Data Identifier Methods

class rucio.client.didclient.DIDClient(rucio_host=None, auth_host=None, account=None, ca_cert=None, auth_type=None, creds=None, timeout=600, user_agent='rucio-clients', vo=None)

Bases: rucio.client.baseclient.BaseClient

DataIdentifier client class for working with data identifiers

ARCHIVES_BASEURL = 'archives'

DIDS_BASEURL = 'dids'

add_container(scope, name, statuses=None, meta=None, rules=None, lifetime=None)

Add data identifier for a container.

Parameters

• scope – The scope name.

• name – The data identifier name.

• statuses – Dictionary with statuses, e.g.g {‘monotonic’:True}.

• meta – Meta-data associated with the data identifier is represented using key/value pairs in a dictionary.

• rules – Replication rules associated with the data identifier. A list of dictionaries, e.g., [{‘copies’: 2, ‘rse_expression’: ‘TIERS1’}, ].

• lifetime – DID’s lifetime (in seconds).

add_containers(cnts)

Bulk add containers.

Parameters

cnts – A list of containers.

add_containers_to_container(scope, name, cnts)

Add containers to container.

Parameters

• scope – The scope name.

• name – The dataset name.

• dsns – The content.

add_containers_to_containers(attachments)

Add containers_to_containers.

Parameters

attachments – The attachments. attachments is: [attachment, attachment, …] attachment is: {‘scope’: scope, ‘name’: name, ‘dids’: dids} dids is: [{‘scope’: scope, ‘name’: name}, …]

add_dataset(scope, name, statuses=None, meta=None, rules=None, lifetime=None, files=None, rse=None)

Add data identifier for a dataset.

Parameters

• scope – The scope name.

• name – The data identifier name.

• statuses – Dictionary with statuses, e.g.g {‘monotonic’:True}.

• lifetime – DID’s lifetime (in seconds).

• files – The content.

• rse – The RSE name when registering replicas.

Rules

Replication rules associated with the data identifier. A list of dictionaries, e.g., [{‘copies’: 2, ‘rse_expression’: ‘TIERS1’}, ].

add_datasets(dsns)

Bulk add datasets.

Parameters

dsns – A list of datasets.

add_datasets_to_container(scope, name, dsns)

Add datasets to container.

Parameters

• scope – The scope name.

• name – The dataset name.

• dsns – The content.

add_datasets_to_containers(attachments)

Add datasets_to_containers.

Parameters

attachments – The attachments. attachments is: [attachment, attachment, …] attachment is: {‘scope’: scope, ‘name’: name, ‘dids’: dids} dids is: [{‘scope’: scope, ‘name’: name}, …]

add_did(scope, name, type, statuses=None, meta=None, rules=None, lifetime=None, dids=None, rse=None)

Add data identifier for a dataset or container.

Parameters

• scope – The scope name.

• name – The data identifier name.

• statuses – Dictionary with statuses, e.g.g {‘monotonic’:True}.

• lifetime – DID’s lifetime (in seconds).

• dids – The content.

• rse – The RSE name when registering replicas.

Paran type

The data identifier type (file|dataset|container).

Rules

Replication rules associated with the data identifier. A list of dictionaries, e.g., [{‘copies’: 2, ‘rse_expression’: ‘TIERS1’}, ].

add_dids(dids)

Bulk add datasets/containers.

add_files_to_archive(scope, name, files)

Add files to archive.

Parameters

• scope – The scope name.

• name – The dataset name.

• files – The content.

add_files_to_dataset(scope, name, files, rse=None)

Add files to datasets.

Parameters

• scope – The scope name.

• name – The dataset name.

• files – The content.

• rse – The RSE name when registering replicas.

add_files_to_datasets(attachments, ignore_duplicate=False)

Add files to datasets.

Parameters

• attachments – The attachments. attachments is: [attachment, attachment, …] attachment is: {‘scope’: scope, ‘name’: name, ‘dids’: dids} dids is: [{‘scope’: scope, ‘name’: name}, …]

• ignore_duplicate – If True, ignore duplicate entries.

add_temporary_dids(dids)

Bulk add temporary data identifiers.

Parameters

dids – A list of dids.

attach_dids(scope, name, dids, rse=None)

Attach data identifier.

Parameters

• scope – The scope name.

• name – The data identifier name.

• dids – The content.

• rse – The RSE name when registering replicas.

attach_dids_to_dids(attachments, ignore_duplicate=False)

Add dids to dids.

Parameters

attachments – The attachments. attachments is: [attachment, attachment, …] attachment is: {‘scope’: scope, ‘name’: name, ‘dids’: dids} dids is: [{‘scope’: scope, ‘name’: name}, …] :param ignore_duplicate: If True, ignore duplicate entries.

close(scope, name)

close dataset/container

Parameters

• scope – The scope name.

• name – The dataset/container name.

create_did_sample(input_scope, input_name, output_scope, output_name, nbfiles)

Create a sample from an input collection.

Parameters

• input_scope – The scope of the input DID.

• input_name – The name of the input DID.

• output_scope – The scope of the output dataset.

• output_name – The name of the output dataset.

• account – The account.

• nbfiles – The number of files to register in the output dataset.

delete_metadata(scope, name, key)

Delete data identifier metadata

Parameters

• scope – The scope name.

• name – The data identifier.

• key – the key.

detach_dids(scope, name, dids)

Detach data identifier

Parameters

• scope – The scope name.

• name – The data identifier name.

• dids – The content.

get_dataset_by_guid(guid)

Get the parent datasets for a given GUID. :param guid: The GUID.

Returns

A did

get_did(scope, name, dynamic=False)

Retrieve a single data identifier.

Parameters

• scope – The scope name.

• name – The data identifier name.

• dynamic – Calculate sizes dynamically when True

get_metadata(scope, name, plugin='DID_COLUMN')

Get data identifier metadata

Parameters

• scope – The scope name.

• name – The data identifier name.

get_metadata_bulk(dids)

Bulk get data identifier metadata :param dids: A list of dids.

list_archive_content(scope, name)

List archive contents.

Parameters

• scope – The scope name.

• name – The data identifier name.

list_associated_rules_for_file(scope, name)

List the associated rules a file is affected from..

Parameters

• scope – The scope name.

• name – The file name.

list_content(scope, name)

List data identifier contents.

Parameters

• scope – The scope name.

• name – The data identifier name.

list_content_history(scope, name)

List data identifier contents history.

Parameters

• scope – The scope name.

• name – The data identifier name.

list_did_rules(scope, name)

List the associated rules of a data identifier.

Parameters

• scope – The scope name.

• name – The data identifier name.

list_dids(scope, filters, type='collection', long=False, recursive=False)

List all data identifiers in a scope which match a given pattern.

Parameters

• scope – The scope name.

• filters – A dictionary of key/value pairs like {‘name’: ‘file_name’,’rse-expression’: ‘tier0’}.

• type – The type of the did: ‘all’(container, dataset or file)|’collection’(dataset or container)|’dataset’|’container’|’file’

• long – Long format option to display more information for each DID.

• recursive – Recursively list DIDs content.

list_dids_by_meta(scope=None, select={})

Gets all dids matching the values of the provided metadata keys :param scope: the scope of the search :param select: the key value pairs to search with(query in json format)

list_dids_extended(scope, filters, type='collection', long=False, recursive=False)

List all data identifiers in a scope which match a given pattern. Extended version that goes through plugin mechanism.

Parameters

• scope – The scope name.

• filters – A dictionary of key/value pairs like {‘name’: ‘file_name’,’rse-expression’: ‘tier0’}.

• type – The type of the did: ‘all’(container, dataset or file)|’collection’(dataset or container)|’dataset’|’container’|’file’

• long – Long format option to display more information for each DID.

• recursive – Recursively list DIDs content.

list_files(scope, name, long=None)

List data identifier file contents.

Parameters

• scope – The scope name.

• name – The data identifier name.

• long – A boolean to choose if GUID is returned or not.

list_parent_dids(scope, name)

List parent dataset/containers of a did.

Parameters

• scope – The scope.

• name – The name.

resurrect(dids)

Resurrect a list of dids.

Parameters

dids – A list of dids [{‘scope’: scope, ‘name’: name}, …]

scope_list(scope, name=None, recursive=False)

List data identifiers in a scope.

Parameters

• scope – The scope name.

• name – The data identifier name.

• recursive – boolean, True or False.

set_metadata(scope, name, key, value, recursive=False)

Set data identifier metadata

Parameters

• scope – The scope name.

• name – The data identifier name.

• key – the key.

• value – the value.

• recursive – Option to propagate the metadata change to content.

set_status(scope, name, **kwargs)

Set data identifier status

Parameters

• scope – The scope name.

• name – The data identifier name.

• kwargs – Keyword arguments of the form status_name=value.