Installing Rucio daemons¶
Prerequisites¶
The Rucio daemons run on Python 2.7, 3.6 and 3.7 on any Unix-like platform.
Install via pip¶
Heads up: We recommend to use the docker-based install (see next section) as it will configure many things for you automatically. Only use the pip-based install if you have a good reason and know how to configure your web service manually:
pip install rucio
This will pull the latest release from PyPi. The Rucio server also needs several Python dependencies. These are all listed in the file tools/pip-requires
and will be pulled in as necessary.
Install via Docker¶
This image provides the Rucio daemons. Each daemon has to be run in a separate container. It supports MySQL, PostgreSQL, Oracle, and SQLite as database backends.
This image expects that there is an already initialised Rucio DB. To start a simple judge-cleaner daemon using a database on mysql.db without an additional parameters just run this:
`docker run --name=rucio-judge-cleaner -e RUCIO_CFG_DATABASE_DEFAULT="mysql+pymysql://rucio:rucio@mysql.db/rucio" -e RUCIO_DAEMON=judge-cleaner rucio/rucio-daemons`
The RUCIO_DAEMON environment variable gives the name of the rucio daemon.
Rucio can be configured fully using environment variables like RUCIO_CFG_DATABASE_DEFAULT. If you want to instead use a complete rucio.cfg it can also be mounted. This will then ignore the RUCIO_CFG environment variables:
The rucio.cfg is used to configure the database backend and the daemons:
`docker run --name=rucio-judge-cleaner -v /tmp/rucio.cfg:/opt/rucio/etc/rucio.cfg -e RUCIO_DAEMON=judge-cleaner rucio/rucio-daemons`
By default the daemon logs are written to stdout and stderr if you want to write to a file you can use RUCIO_ENABLE_LOGS like this:
`docker run --name=rucio-judge-cleaner -v /tmp/rucio.cfg:/opt/rucio/etc/rucio.cfg -v /tmp/logs:/var/log/rucio -e RUCIO_DAEMON=judge-cleaner -e RUCIO_ENABLE_LOGS=True rucio/rucio-daemons`
Environment Variables¶
As shown in the examples above the rucio-daemon image can be configured using environment variables that are passed with docker run. Below is a list of all available variables and their behaviour:
RUCIO_DAEMON¶
This variable is mandatory and it specifies the name of the daemon, e.g., hermes, kronos, judge-evaluator, etc.
RUCIO_DAEMON_ARGS¶
Any additional command line parameter can be specified here, e.g., –run-once. This field is optional.
RUCIO_ENABLE_LOGFILE¶
By default, the log output of the daemon is written to stdout and stderr. If you set this variable to True the output will be written to access_log and error_log under /var/log/rucio.
RUCIO_CFG configuration parameters:¶
Environment variables can be used to set values for the auto-generated rucio.cfg. The names are derived from the actual names in the configuration file prefixed by RUCIO_CFG, e.g., the default value in the database section becomes RUCIO_CFG_DATABASE_DEFAULT. All available environment variables are:
RUCIO_CFG_ACCOUNTS_SPECIAL_ACCOUNTS
RUCIO_CFG_COMMON_LOGDIR
RUCIO_CFG_COMMON_LOGLEVEL
RUCIO_CFG_COMMON_MAILTEMPLATEDIR
RUCIO_CFG_DATABASE_DEFAULT
RUCIO_CFG_DATABASE_SCHEMA
RUCIO_CFG_DATABASE_SCHEMA
RUCIO_CFG_DATABASE_POOL_RESET_ON_RETURN
RUCIO_CFG_DATABASE_ECHO
RUCIO_CFG_DATABASE_POOL_RECYCLE
RUCIO_CFG_DATABASE_POOL_SIZE
RUCIO_CFG_DATABASE_POOL_TIMEOUT
RUCIO_CFG_DATABASE_MAX_OVERFLOW
RUCIO_CFG_DATABASE_POWUSERACCOUNT
RUCIO_CFG_DATABASE_POWUSERPASSWORD
RUCIO_CFG_MONITOR_CARBON_SERVER
RUCIO_CFG_MONITOR_CARBON_PORT
RUCIO_CFG_MONITOR_USER_SCOPE
RUCIO_CFG_PERMISSION_POLICY
RUCIO_CFG_PERMISSION_SCHEMA
RUCIO_CFG_PERMISSION_LFN2PFN_ALGORITHM_DEFAULT
RUCIO_CFG_PERMISSION_SUPPORT
RUCIO_CFG_PERMISSION_SUPPORT_RUCIO
RUCIO_CFG_AUTOMATIX_SITES
RUCIO_CFG_AUTOMATIX_SLEEP_TIME
RUCIO_CFG_AUTOMATIX_DATASET_LIFETIME
RUCIO_CFG_AUTOMATIX_SET_METADATA
RUCIO_CFG_AUDITOR_RESULTS
RUCIO_CFG_AUDITOR_CACHE
RUCIO_CFG_CONVEYOR_SCHEME
RUCIO_CFG_CONVEYOR_TRANSFERTOOL
RUCIO_CFG_CONVEYOR_FTSHOSTS
RUCIO_CFG_CONVEYOR_CACERT
RUCIO_CFG_CONVEYOR_USERCERT
RUCIO_CFG_CONVEYOR_CACHE_TIME
RUCIO_CFG_CONVEYOR_USE_DETERMINISTIC_ID
RUCIO_CFG_CONVEYOR_POLL_TIMEOUT
RUCIO_CFG_CONVEYOR_SUBMIT_TIMEOUT
RUCIO_CFG_CONVEYOR_BRING_ONLINE
RUCIO_CFG_CONVEYOR_QUEUE_MODE
RUCIO_CFG_CONVEYOR_USING_MEMCACHE
RUCIO_CFG_CONVEYOR_FTSMONHOSTS
RUCIO_CFG_MESSAGING_FTS3_PORT
RUCIO_CFG_MESSAGING_FTS3_SSL_KEY_FILE
RUCIO_CFG_MESSAGING_FTS3_SSL_CERT_FILE
RUCIO_CFG_MESSAGING_FTS3_DESTINATION
RUCIO_CFG_MESSAGING_FTS3_BROKERS
RUCIO_CFG_MESSAGING_FTS3_VONAME
RUCIO_CFG_MESSAGING_HERMES_USERNAME
RUCIO_CFG_MESSAGING_HERMES_PASSWORD
RUCIO_CFG_MESSAGING_HERMES_PORT
RUCIO_CFG_MESSAGING_HERMES_NONSSL_PORT
RUCIO_CFG_MESSAGING_HERMES_USE_SSL
RUCIO_CFG_MESSAGING_HERMES_SSL_KEY_FILE
RUCIO_CFG_MESSAGING_HERMES_SSL_CERT_FILE
RUCIO_CFG_MESSAGING_HERMES_DESTINATION
RUCIO_CFG_MESSAGING_HERMES_BROKERS
RUCIO_CFG_MESSAGING_HERMES_VONAME
RUCIO_CFG_MESSAGING_HERMES_EMAIL_FROM
RUCIO_CFG_MESSAGING_HERMES_EMAIL_TEST
RUCIO_CFG_TRACER_KRONOS_BROKERS
RUCIO_CFG_TRACER_KRONOS_PORT
RUCIO_CFG_TRACER_SSL_KEY_FILE
RUCIO_CFG_TRACER_SSL_CERT_FILE
RUCIO_CFG_TRACER_QUEUE
RUCIO_CFG_TRACER_PREFETCH_SIZE
RUCIO_CFG_TRACER_CHUNKSIZE
RUCIO_CFG_TRACER_SUBSCRIPTION_ID
RUCIO_CFG_TRACER_USE_SSL
RUCIO_CFG_TRACER_RECONNECT_ATTEMPTS
RUCIO_CFG_TRACER_EXCLUDED_USRDNS
RUCIO_CFG_TRACER_KRONOS_USERNAME
RUCIO_CFG_TRACER_KRONOS_PASSWORD
RUCIO_CFG_TRACER_DATASET_WAIT
RUCIO_CFG_MESSAGING_CACHE_PORT
RUCIO_CFG_MESSAGING_CACHE_SSL_KEY_FILE
RUCIO_CFG_MESSAGING_CACHE_SSL_CERT_FILE
RUCIO_CFG_MESSAGING_CACHE_DESTINATION
RUCIO_CFG_MESSAGING_CACHE_BROKERS
RUCIO_CFG_MESSAGING_CACHE_VONAME
RUCIO_CFG_MESSAGING_CACHE_ACCOUNT
RUCIO_CFG_CREDENTIALS_GCS
RUCIO_CFG_CREDENTIALS_SIGNATURE_LIFETIME