Installing Rucio daemons

Prerequisites

The Rucio daemons run on Python 2.7, 3.6 and 3.7 on any Unix-like platform.

Install via pip

Heads up: We recommend to use the docker-based install (see next section) as it will configure many things for you automatically. Only use the pip-based install if you have a good reason and know how to configure your web service manually:

pip install rucio

This will pull the latest release from PyPi. The Rucio server also needs several Python dependencies. These are all listed in the file tools/pip-requires and will be pulled in as necessary.

Install via Docker

This image provides the Rucio daemons. Each daemon has to be run in a separate container. It supports MySQL, PostgreSQL, Oracle, and SQLite as database backends.

This image expects that there is an already initialised Rucio DB. To start a simple judge-cleaner daemon using a database on mysql.db without an additional parameters just run this:

`docker run --name=rucio-judge-cleaner -e RUCIO_CFG_DATABASE_DEFAULT="mysql+pymysql://rucio:rucio@mysql.db/rucio" -e RUCIO_DAEMON=judge-cleaner  rucio/rucio-daemons`

The RUCIO_DAEMON environment variable gives the name of the rucio daemon.

Rucio can be configured fully using environment variables like RUCIO_CFG_DATABASE_DEFAULT. If you want to instead use a complete rucio.cfg it can also be mounted. This will then ignore the RUCIO_CFG environment variables:

The rucio.cfg is used to configure the database backend and the daemons:

`docker run --name=rucio-judge-cleaner -v /tmp/rucio.cfg:/opt/rucio/etc/rucio.cfg -e RUCIO_DAEMON=judge-cleaner  rucio/rucio-daemons`

By default the daemon logs are written to stdout and stderr if you want to write to a file you can use RUCIO_ENABLE_LOGS like this:

`docker run --name=rucio-judge-cleaner -v /tmp/rucio.cfg:/opt/rucio/etc/rucio.cfg -v /tmp/logs:/var/log/rucio -e RUCIO_DAEMON=judge-cleaner  -e RUCIO_ENABLE_LOGS=True rucio/rucio-daemons`

Environment Variables

As shown in the examples above the rucio-daemon image can be configured using environment variables that are passed with docker run. Below is a list of all available variables and their behaviour:

RUCIO_DAEMON

This variable is mandatory and it specifies the name of the daemon, e.g., hermes, kronos, judge-evaluator, etc.

RUCIO_DAEMON_ARGS

Any additional command line parameter can be specified here, e.g., –run-once. This field is optional.

RUCIO_ENABLE_LOGFILE

By default, the log output of the daemon is written to stdout and stderr. If you set this variable to True the output will be written to access_log and error_log under /var/log/rucio.

RUCIO_CFG configuration parameters:

Environment variables can be used to set values for the auto-generated rucio.cfg. The names are derived from the actual names in the configuration file prefixed by RUCIO_CFG, e.g., the default value in the database section becomes RUCIO_CFG_DATABASE_DEFAULT. All available environment variables are:

  • RUCIO_CFG_ACCOUNTS_SPECIAL_ACCOUNTS

  • RUCIO_CFG_COMMON_LOGDIR

  • RUCIO_CFG_COMMON_LOGLEVEL

  • RUCIO_CFG_COMMON_MAILTEMPLATEDIR

  • RUCIO_CFG_DATABASE_DEFAULT

  • RUCIO_CFG_DATABASE_SCHEMA

  • RUCIO_CFG_DATABASE_SCHEMA

  • RUCIO_CFG_DATABASE_POOL_RESET_ON_RETURN

  • RUCIO_CFG_DATABASE_ECHO

  • RUCIO_CFG_DATABASE_POOL_RECYCLE

  • RUCIO_CFG_DATABASE_POOL_SIZE

  • RUCIO_CFG_DATABASE_POOL_TIMEOUT

  • RUCIO_CFG_DATABASE_MAX_OVERFLOW

  • RUCIO_CFG_DATABASE_POWUSERACCOUNT

  • RUCIO_CFG_DATABASE_POWUSERPASSWORD

  • RUCIO_CFG_MONITOR_CARBON_SERVER

  • RUCIO_CFG_MONITOR_CARBON_PORT

  • RUCIO_CFG_MONITOR_USER_SCOPE

  • RUCIO_CFG_PERMISSION_POLICY

  • RUCIO_CFG_PERMISSION_SCHEMA

  • RUCIO_CFG_PERMISSION_LFN2PFN_ALGORITHM_DEFAULT

  • RUCIO_CFG_PERMISSION_SUPPORT

  • RUCIO_CFG_PERMISSION_SUPPORT_RUCIO

  • RUCIO_CFG_AUTOMATIX_SITES

  • RUCIO_CFG_AUTOMATIX_SLEEP_TIME

  • RUCIO_CFG_AUTOMATIX_DATASET_LIFETIME

  • RUCIO_CFG_AUTOMATIX_SET_METADATA

  • RUCIO_CFG_AUDITOR_RESULTS

  • RUCIO_CFG_AUDITOR_CACHE

  • RUCIO_CFG_CONVEYOR_SCHEME

  • RUCIO_CFG_CONVEYOR_TRANSFERTOOL

  • RUCIO_CFG_CONVEYOR_FTSHOSTS

  • RUCIO_CFG_CONVEYOR_CACERT

  • RUCIO_CFG_CONVEYOR_USERCERT

  • RUCIO_CFG_CONVEYOR_CACHE_TIME

  • RUCIO_CFG_CONVEYOR_USE_DETERMINISTIC_ID

  • RUCIO_CFG_CONVEYOR_POLL_TIMEOUT

  • RUCIO_CFG_CONVEYOR_SUBMIT_TIMEOUT

  • RUCIO_CFG_CONVEYOR_BRING_ONLINE

  • RUCIO_CFG_CONVEYOR_QUEUE_MODE

  • RUCIO_CFG_CONVEYOR_USING_MEMCACHE

  • RUCIO_CFG_CONVEYOR_FTSMONHOSTS

  • RUCIO_CFG_MESSAGING_FTS3_PORT

  • RUCIO_CFG_MESSAGING_FTS3_SSL_KEY_FILE

  • RUCIO_CFG_MESSAGING_FTS3_SSL_CERT_FILE

  • RUCIO_CFG_MESSAGING_FTS3_DESTINATION

  • RUCIO_CFG_MESSAGING_FTS3_BROKERS

  • RUCIO_CFG_MESSAGING_FTS3_VONAME

  • RUCIO_CFG_MESSAGING_HERMES_USERNAME

  • RUCIO_CFG_MESSAGING_HERMES_PASSWORD

  • RUCIO_CFG_MESSAGING_HERMES_PORT

  • RUCIO_CFG_MESSAGING_HERMES_NONSSL_PORT

  • RUCIO_CFG_MESSAGING_HERMES_USE_SSL

  • RUCIO_CFG_MESSAGING_HERMES_SSL_KEY_FILE

  • RUCIO_CFG_MESSAGING_HERMES_SSL_CERT_FILE

  • RUCIO_CFG_MESSAGING_HERMES_DESTINATION

  • RUCIO_CFG_MESSAGING_HERMES_BROKERS

  • RUCIO_CFG_MESSAGING_HERMES_VONAME

  • RUCIO_CFG_MESSAGING_HERMES_EMAIL_FROM

  • RUCIO_CFG_MESSAGING_HERMES_EMAIL_TEST

  • RUCIO_CFG_TRACER_KRONOS_BROKERS

  • RUCIO_CFG_TRACER_KRONOS_PORT

  • RUCIO_CFG_TRACER_SSL_KEY_FILE

  • RUCIO_CFG_TRACER_SSL_CERT_FILE

  • RUCIO_CFG_TRACER_QUEUE

  • RUCIO_CFG_TRACER_PREFETCH_SIZE

  • RUCIO_CFG_TRACER_CHUNKSIZE

  • RUCIO_CFG_TRACER_SUBSCRIPTION_ID

  • RUCIO_CFG_TRACER_USE_SSL

  • RUCIO_CFG_TRACER_RECONNECT_ATTEMPTS

  • RUCIO_CFG_TRACER_EXCLUDED_USRDNS

  • RUCIO_CFG_TRACER_KRONOS_USERNAME

  • RUCIO_CFG_TRACER_KRONOS_PASSWORD

  • RUCIO_CFG_TRACER_DATASET_WAIT

  • RUCIO_CFG_MESSAGING_CACHE_PORT

  • RUCIO_CFG_MESSAGING_CACHE_SSL_KEY_FILE

  • RUCIO_CFG_MESSAGING_CACHE_SSL_CERT_FILE

  • RUCIO_CFG_MESSAGING_CACHE_DESTINATION

  • RUCIO_CFG_MESSAGING_CACHE_BROKERS

  • RUCIO_CFG_MESSAGING_CACHE_VONAME

  • RUCIO_CFG_MESSAGING_CACHE_ACCOUNT

  • RUCIO_CFG_CREDENTIALS_GCS

  • RUCIO_CFG_CREDENTIALS_SIGNATURE_LIFETIME