Installation

A usual installation of Daiquiri contains of a set of different components:

  1. A directory holding all the settings and customisations, custom to your installation of Daiquiri. We will call this directory app, but you can use any name you see fit. This app directory corresponds to a project in Django terms.
  2. The django-daiquiri library, which is centrally maintained, and which is installed as a packacge using pip.
  3. The application database to store the content, which is generated by the users of your Daiquiri installation. Currently, we support PostgreSQL and MySQL (deprecated).
  4. A (bigger) scientific database, where the actual science data is stored. This can be PostgreSQL and MySQL (deprecated).
  5. A directory on your filesystem with static files to be downloaded.
  6. A RabbitMQ installation as message broker for the asyncronous queues.

For testing and development, you can run Daiquiri using your regular user account. On a production system, a dedicated user account should be used. For this documentation we will use a user called daiquiri with the group daiquiri and the home directory /srv/daiquiri. This user can be created with:

useradd -m -d /srv/daiquiri -c "Daiquiri user" -s /bin/bash daiquiri

Do not use the root user to run Daiquiri! It is a bad idea anyway and several steps of the installation will not work. sudo is used in the installation when requiring root-privileges to install packages.

Prerequisites

Although, most dependencies are installed from the Python Package Index, some dependecies need to be installed using the operation systems packages manager. Here we document the minimum prerequisites:

# Centos
yum install -y \
    epel-release \
    git \
    gcc gcc-c++ \
    libxml2-devel libxslt-devel \
    openssl-devel \
    python3-devel

# Debian or Ubuntu
apt-get install -y \
    git \
    build-essential \
    libxml2-dev libxslt-dev \
    zlib1g-dev \
    libssl-dev \
    python3-dev \
    python3-venv

Obtaining the app directory

The next step is to create the app directory by cloning the corresponding repository.

git clone https://github.com/django-daiquiri/app

Note that this is not the main django-daiquiri repository, only the configuration files. Inside this directory, you will find:

  • a config directory, containing the main settings of your Daiquiri installation,
  • a manage.py script, which is the main way to interact with your Daiquiri installation on the command line. Most of the following steps will use this script.

Install Python packages

After you have obtained the app, you need to install the django-daiquiri package and the other python dependencies. Change to the app directory and create a Virtual Environment (this is done as your user or the created daiquiri user, not as root):

cd app

python3 -m venv env
source env/bin/activate

pip install --upgrade pip setuptools wheel

After the virtual environment is activated, the django-daiquiri package can be installed using pip:

pip install django-daiquiri

If you want to install the current master branch directly from GutHub, you alternatively use:

pip install git+https://github.com/django-daiquiri/daiquiri

The virtual environment encapsulates your Daiquiri installation from the rest of the system. This makes it possible to run several applications with different python dependencies on one machine and to install the dependencies without root permissions.

Important: The virtual enviroment needs to be activated, using source env/bin/activate everytime a new terminal is used. This can be automated using your .bashrc.

Basic setup

The settings of a Daiquiri application are specified in two files:

  • config/settings/base.py should be part of the app repository and holds the basic settings for this particular site.
  • .env is excluded from the repository and should contain (possible secret information) about this machine. This file will be different on the development and the production system.

To set up the application, you need to create a new file .env in your cloned app directory.

You can use .env.sample as template, i.e.:

cp .env.sample .env

The different settings are explained in detail later in the documentation. For a minimal configuration, you need to set

SECRET_KEY=<a secret random string>

SITE_URL=http://localhost:8000

DEBUG=True
ASYNC=False

DATABASE_DEFAULT=postgresql://<user>:<pass>@<host>/<db>
DATABASE_DATA=postgresql://<user>:<pass>@<host>/<db>

FILES_BASE_PATH=files
QUERY_DOWNLOAD_DIR=download
QUERY_UPLOAD_DIR=upload

Databases

As mentioned earlier, Daiquiri uses two separate database connections, one for the web application (default) and one for the scientific data (data). These database can be on different machines and you can even use MySQL for one and PostgreSQL for the other. For an SQLite3 database use sqlite:///database.sqlite3 to create the database in the app directory. For PostgreSQL and peer auth use postgresql://@/<db>.

Note: The queryparser has dropped the active support of MySQL, thus we strongly recommend to use PostgreSQL.

In order to use the different database connections, install the corresponding package with pip:

pip install psycopg[binary]  # for PostgreSQL
pip install mysqlclient      # for MySQL or MariaDB

Note: For the full ADQL support, it is required to install the pg_sphere extension on the data PostgreSQL database. It is not required for the default database.

In your virtualenv, you can get the, which you need to perform on your database with the sqlcreate command:

python manage.py sqlcreate

Initialization

After editing the settings, initialize the application using:

python manage.py migrate                  # initializes the web database
python manage.py migrate --database tap   # initializes the tap schema in the scientific db
python manage.py migrate --database oai   # initializes the oai schema in the scientific db
python manage.py createsuperuser          # creates an admin user
python manage.py download_vendor_files    # dowloads front-end files from the CDN

After these steps, Daiquiri can be run using Django’s intergrated development server:

python manage.py runserver

Then, Daiquiri is available on http://127.0.0.1:8000 in your (local) browser.

Asyncronous tasks

Several tasks in Daiquiri can make use of asyncronous tasks (e.g. scientic database queries). Daiquiri can be used without asyncronous tasks, but for any deployment setup this functionality is recomended. RabbiMQ needs to be installed for asyncronous tasks to work. For Debian and Ubuntu, RabbitMQ can be installed from the distribution:

apt-get install rabbitmq-server

For Centos 7/8, the offical repo needs to be installed as well as its erlang dependencies. This can be done from packagecloud.io:

# see also https://www.rabbitmq.com/install-rpm.html
curl -s https://packagecloud.io/install/repositories/rabbitmq/erlang/script.rpm.sh | sudo bash
curl -s https://packagecloud.io/install/repositories/rabbitmq/rabbitmq-server/script.rpm.sh | sudo bash
yum install rabbitmq-server

systemctl start rabbitmq-server
systemctl enable rabbitmq-server

The asyncronous tasks can be activated the .env file of your Daiquiri application:

ASYNC=True

For a more complicated RabbitMQ setup the CELERY_BROKER_URL setting is used as explained here.

Daiquiri uses 3 different workers:

  • default: For miscanelous database queries performed by users.
  • query: For asynconous database queriesperformed by users.
  • download: For the server-side creation of download files and zip archives.

In a development setup these workers can be started using:

python manage.py runworker default
python manage.py runworker query
python manage.py runworker download

in your virtual app.