`datafed.CLI`

Module Contents

Classes

`_AliasedGroup`	A group allows a command to have subcommands attached. This is
`_AliasedGroupRoot`	A group allows a command to have subcommands attached. This is

Functions

`run`()
`init`([opts])
`loginByPassword`(uid, password)
`loginByToken`(token)
`command`(command)
`_set_script_cb`(ctx, param, value)
`_set_verbosity_cb`(ctx, param, value)
`_global_context_options`(func)
`_global_output_options`(func)
`_cli`(ctx, args, *kwargs)	'datafed' is the command-line interface (CLI) for the DataFed federated data management
`_genDoc`(ctx)
`_genDocHeader`(cmd, level)
`_genDocCmd`(cmd, ctx, level[, parname, recurse])
`_wc`(coll_id)	Set/print current working collection or path. 'ID' can be a collection ID, alias, user
`_wp`()	Get current working path. Displays the full path of the current working
`_data`()
`_dataView`(data_id, context)	View data record information. Displays record title, description, tags,
`_dataCreate`(title, alias, description, tags, ...)	Create a new data record. The data record 'title' is required, but all
`_dataUpdate`(data_id, title, alias, description, tags, ...)	Update an existing data record. The data record ID is required and can be
`_dataDelete`(data_id, force, context)	Delete one or more existing data records. Multiple ID arguments can be
`_dataGet`(df_id, path, wait, encrypt, orig_fname, context)	Get (download) raw data of data records and/or collections. Multiple ID
`_dataPut`(data_id, path, wait, extension, encrypt, context)	Put (upload) raw data located at PATH to DataFed record ID. The ID
`_batch`()
`_data_batch_create`(collection, file, context)	Batch create data records from JSON file(s). Multiple FILE arguments may be
`_data_batch_update`(file)	Batch update data records from JSON file(s). Multiple FILE arguments may be
`_list`(ctx, item_id, offset, count, context)	List contents of a collection, or shared items. ID may be a collection ID
`_coll`()
`_collView`(coll_id, context)	View collection information. Displays collection title, description, and
`_collCreate`(title, alias, description, tags, topic, ...)	Create a new collection. The collection 'title' is required, but all
`_collUpdate`(coll_id, title, alias, description, tags, ...)	Update an existing collection. The collection ID is required and can be
`_collDelete`(coll_id, force, context)	Delete one or more existing collections. Multiple ID arguments can be
`_collItemsAdd`(coll_id, item_id, context)	Add data records and/or collections to a collection. COLL_ID is the
`_coll_rem`(coll_id, item_id, context)	Remove data records and/or collections from a collection. COLL_ID is the
`_query`(args, *kwargs)
`_queryList`(offset, count)	List saved queries.
`_queryView`(qry_id)	View a saved query by ID.
`_queryCreate`(title, coll_mode, coll, id, text, tag, ...)	Create a saved query.
`_queryUpdate`(qry_id, title, coll_mode, coll, id, text, ...)	Update a saved query. The title and search terms of a query may be updated;
`_queryDelete`(qry_id)	Delete a saved query by ID.
`_queryExec`(qry_id, offset, count)	Execute a saved query by ID.
`_queryRun`(coll_mode, coll, id, text, tag, schema, ...)	Run a direct query on data or collections. The default scope is the current
`_user`()
`_userListCollab`(offset, count)	List all users that are collaborators. Collaborators are defined as users
`_userListAll`(offset, count)	List all users.
`_userView`(uid)	View user information.
`_userWho`()	Show current authenticated user ID.
`_project`()
`_projectList`(owned, admin, member, offset, count)	List projects associated with current user. List projects that are owned or managed by the
`_projectView`(proj_id)	View project information. Current user must have a role (owner, manager, or
`_shares`(users, projects)	List users and/or projects sharing data with current user.
`_task`()
`_taskList`(time_from, to, since, status, offset, count)	List recent tasks. If no time or status filter options are
`_taskView`(task_id)	Show task information. Use the ID argument to view a specific task
`_ep`()
`_epGet`()	Get Globus endpoint for the current session. At the start of a session, the
`_epSet`(endpoint)	Set endpoint for the current session. If no endpoint is given, the
`_epList`()	List recently used endpoints.
`_epDefault`()
`_epDefaultGet`()	Show the default Globus endpoint.
`_epDefaultSet`(current, endpoint)	Set the default Globus endpoint. The default endpoint will be set from the
`_setup`(ctx)	Setup local credentials. This command installs DataFed credentials for the
`_verbositySet`(level)	Set/display verbosity level. The verbosity level argument can be 0
`_help_cli`(ctx, command)	Show DataFed CLI help. Include a command name as the argument to see
`_exit_cli`()	Exit an interactive session. Ctrl-C may also be used to exit the shell.
`_print_msg`(level, message[, err])
`_print_ack_reply`([reply])
`_print_listing`(message)
`_print_user_listing`(message)
`_print_proj_listing`(message)
`_print_endpoints`(message)
`_print_data`(message)
`_print_batch`(message)
`_print_coll`(message)
`_print_deps`(dr)
`_print_task_listing`(message)
`_print_task`(message)
`_print_task_array`(message)
`_print_user`(message)
`_print_proj`(message)
`_print_path`(message)
`_print_query`(message)
`_wrap_text`(text, prefix, indent[, compact])
`_resolve_id`(df_id)
`_resolve_coll_id`(coll_id[, context])
`_generic_reply_handler`(reply, printFunc)
`_setWorkingCollectionTitle`()
`_arrayToCSV`(items, skip)
`_arrayToDotted`(items[, skip])
`_printJSON`(json, cur_indent, indent)
`_printJSON_List`(json, cur_indent, indent)
`_bar_adaptive_human_readable`(current, total[, width])
`_initialize`(opts)
`_addConfigOptions`()

Attributes

`_OM_TEXT`
`_OM_JSON`
`_OM_RETN`
`_STAT_OK`
`_STAT_ERROR`
`_capi`
`_return_val`
`_uid`
`_cur_ctx`
`_cur_coll`
`_cur_coll_prefix`
`_cur_coll_title`
`_cur_alias_prefix`
`_prev_coll`
`_prev_ctx`
`_list_items`
`_interactive`
`_verbosity_sticky`
`_verbosity`
`_output_mode_sticky`
`_output_mode`
`_ctxt_settings`
`_task_statuses`
`_task_types`
`_initialized`
`_devnull`
`_hdr_lev_char`
`__global_context_options`
`__global_output_options`

datafed.CLI._OM_TEXT = 0

datafed.CLI._OM_JSON = 1

datafed.CLI._OM_RETN = 2

datafed.CLI._STAT_OK = 0

datafed.CLI._STAT_ERROR = 1

datafed.CLI._capi

datafed.CLI._return_val

datafed.CLI._uid

datafed.CLI._cur_ctx

datafed.CLI._cur_coll

datafed.CLI._cur_coll_prefix = 'root'

datafed.CLI._cur_coll_title

datafed.CLI._cur_alias_prefix = ''

datafed.CLI._prev_coll = 'root'

datafed.CLI._prev_ctx

datafed.CLI._list_items = []

datafed.CLI._interactive = True

datafed.CLI._verbosity_sticky = 1

datafed.CLI._verbosity = 1

datafed.CLI._output_mode_sticky

datafed.CLI._output_mode

datafed.CLI._ctxt_settings

datafed.CLI._task_statuses

datafed.CLI._task_types

datafed.CLI._initialized = False

datafed.CLI._devnull

datafed.CLI._hdr_lev_char = ['-', '-', '^', ',']

datafed.CLI.run()

datafed.CLI.init(opts={})

datafed.CLI.loginByPassword(uid, password)

datafed.CLI.loginByToken(token)

datafed.CLI.command(command)

class datafed.CLI._AliasedGroup(name: str | None = None, commands: Dict[str, Command] | Sequence[Command] | None = None, **attrs: Any)

Bases: click.Group

A group allows a command to have subcommands attached. This is the most common way to implement nesting in Click.

Parameters:

name – The name of the group command.
commands – A dict mapping names to Command objects. Can also be a list of Command, which will use Command.name to create the dict.
attrs – Other command arguments described in MultiCommand, Command, and BaseCommand.

Changed in version 8.0: The commmands argument can be a list of command objects.

get_command(ctx, cmd_name): Given a context and a command name, this returns a Command object if it exists or returns None.

resolve_command(ctx, args)

class datafed.CLI._AliasedGroupRoot(name: str | None = None, commands: Dict[str, Command] | Sequence[Command] | None = None, **attrs: Any)

Bases: _AliasedGroup

A group allows a command to have subcommands attached. This is the most common way to implement nesting in Click.

Parameters:

name – The name of the group command.
commands – A dict mapping names to Command objects. Can also be a list of Command, which will use Command.name to create the dict.
attrs – Other command arguments described in MultiCommand, Command, and BaseCommand.

Changed in version 8.0: The commmands argument can be a list of command objects.

get_command(ctx, cmd_name): Given a context and a command name, this returns a Command object if it exists or returns None.

exception datafed.CLI._NoCommand(*args, **kwargs)

Bases: Exception

Common base class for all non-exit exceptions.

datafed.CLI._set_script_cb(ctx, param, value)

datafed.CLI._set_verbosity_cb(ctx, param, value)

datafed.CLI.__global_context_options

datafed.CLI._global_context_options(func)

datafed.CLI.__global_output_options

datafed.CLI._global_output_options(func)

datafed.CLI._cli(ctx, *args, **kwargs)

‘datafed’ is the command-line interface (CLI) for the DataFed federated data management service and may be used to access many of the features available via the DataFed web portal. This CLI may be used interactively (human-friendly output) or for scripting (JSON output) by specifying the -s option.

When the datafed CLI is run without any command arguments, a interactive shell session is started. While in the shell, commands should be entered without specifying the ‘datafed’ prefix.

datafed.CLI._genDoc(ctx)

datafed.CLI._genDocHeader(cmd, level)

datafed.CLI._genDocCmd(cmd, ctx, level, parname=None, recurse=True)

datafed.CLI._wc(coll_id)

Set/print current working collection or path. ‘ID’ can be a collection ID, alias, user or project ID, listing index, previous collection (‘-‘), or path (‘..’,’/’,’~’). ‘cd’ is an alias for the ‘wc’ command.

The ‘wc’ command can be used to switch to a different user or project context by either specifying a user/project ID/alias as the argument, or by specifying a collection ID/alias that is owned by another user or project. In either case, if permission is granted, the CLI context will be changed to the associated user/project and all subsequent commands will act within that context. To return to the authenticated users context, use the ‘wc ~’ command. The ‘~’ path indicates the authenticated user’s root collection; whereas the ‘/’ path is the root colelction of the current context.

datafed.CLI._wp(): Get current working path. Displays the full path of the current working collection starting from the root collection of the associated user or project.

datafed.CLI._data()

datafed.CLI._dataView(data_id, context): View data record information. Displays record title, description, tags, and other informational and administrative fields. ID may be a data record identifier, alias, or index value from a listing. By default, description text is truncated and metadata is not shown unless the verbosity is as level 2.

datafed.CLI._dataCreate(title, alias, description, tags, raw_data_file, extension, external, metadata, metadata_file, schema, schema_enforce, parent, repository, deps, context): Create a new data record. The data record ‘title’ is required, but all other attributes are optional. On success, the ID of the created data record is returned. Note that if a parent collection is specified, and that collection belongs to a project or other collaborator, the creating user must have permission to write to that collection. The raw-data-file option is only supported in interactive mode and is provided as a convenience to avoid a separate dataPut() call.

datafed.CLI._dataUpdate(data_id, title, alias, description, tags, raw_data_file, extension, metadata, metadata_file, metadata_set, schema, schema_enforce, deps_add, deps_rem, context): Update an existing data record. The data record ID is required and can be an ID, alias, or listing index; all other record attributes are optional. The raw-data-file option is only supported in interactive mode and is provided as a convenience to avoid a separate dataPut() call.

datafed.CLI._dataDelete(data_id, force, context): Delete one or more existing data records. Multiple ID arguments can be provided and may data record IDs, aliases, or index values from a listing. By default, a confirmation prompt is used, but this can be bypassed with the ‘–force’ option.

datafed.CLI._dataGet(df_id, path, wait, encrypt, orig_fname, context)

Get (download) raw data of data records and/or collections. Multiple ID arguments can be specified and may be data record and/or collection IDs, aliases, or index values from s listing. The PATH argument is the destination for the download and can be either a full Globus path (with endpoint), or a local file system path (absolute or relative).

If no endpoint is specified in the PATH argument, the current endpoint will be used. If the destination PATH doesn’t exist, it will be created given sufficient filesystem permissions. Note that the path does not have to be local to where the CLI is running - data can be transferred to/from remote systems; however, full Globus paths must used in this case.

datafed.CLI._dataPut(data_id, path, wait, extension, encrypt, context): Put (upload) raw data located at PATH to DataFed record ID. The ID argument may be data record ID, alias, or index value from a listing. The PATH argument specifies the source file for the upload and can be either a full Globus path (with endpoint), or a local file system path (absolute or relative). If no endpoint is specified in the PATH argument, the current endpoint will be used.

datafed.CLI._batch()

datafed.CLI._data_batch_create(collection, file, context): Batch create data records from JSON file(s). Multiple FILE arguments may be specified and are absolute or relative paths to JSON inputs file on a local filesystem. JSON input files may contain individual JSON objects, or arrays of JSON objects. Each JSON object represents a new data record and the JSON must comply with the DataFed record input schema (see online documentation).

datafed.CLI._data_batch_update(file): Batch update data records from JSON file(s). Multiple FILE arguments may be specified and are absolute or relative paths to JSON inputs file on a local filesystem. JSON input files may contain individual JSON objects, or arrays of JSON objects. Each JSON object represents a new data record and the JSON must comply with the DataFed record input schema (see online documentation).

datafed.CLI._list(ctx, item_id, offset, count, context)

List contents of a collection, or shared items. ID may be a collection ID or alias, a path (‘..’,’/’,’~’), a user or project ID, an index value from a listing, or omitted for the current working collection. If the ID is a user or project, the ls command will list shared items associated with the given user or project.

Note: the ‘/’ path lists the root collection of the current context (user or project); whereas the ‘~’ path always lists the root collection of the authenticated user, regardless of context.

datafed.CLI._coll()

datafed.CLI._collView(coll_id, context): View collection information. Displays collection title, description, and other administrative fields. ID may be a collection identifier, alias, or index value from a listing. Use ‘coll list’ command to see items contained in a collection.

datafed.CLI._collCreate(title, alias, description, tags, topic, parent, context): Create a new collection. The collection ‘title’ is required, but all other attributes are optional. On success, the ID of the created collection is returned. Note that if a parent collection is specified, and that collection belongs to a project or other collaborator, the creating user must have permission to write to that collection.

datafed.CLI._collUpdate(coll_id, title, alias, description, tags, topic, context): Update an existing collection. The collection ID is required and can be an ID, alias, or listing index; all other collection attributes are optional.

datafed.CLI._collDelete(coll_id, force, context)

Delete one or more existing collections. Multiple ID arguments can be provided and may be collection IDs, aliases, or index values from a listing. By default, a confirmation prompt is used, but this can be bypassed with the ‘–force’ option.

When a collection is deleted, all contained collections are also deleted; however, contained data records are only deleted if they are not linked to another collection not involved in the deletion.

datafed.CLI._collItemsAdd(coll_id, item_id, context): Add data records and/or collections to a collection. COLL_ID is the destination collection and ITEM_IDs specify one or more data records and/or collections to add to the destination collection. COLL_ID and ITEM_IDs may be IDs, aliases, or index values from a listing. COLL_ID may also be a relative collection path (‘.’, ‘..’, or ‘/’).

datafed.CLI._coll_rem(coll_id, item_id, context): Remove data records and/or collections from a collection. COLL_ID is the containing collection and ITEM_IDs specify one or more data records and/or collections to remove from the containing collection. COLL_ID and ITEM_IDs may be IDs, aliases, or index values from a listing. COLL_ID may also be a relative collection path (‘.’, ‘..’, or ‘/’).

datafed.CLI._query(*args, **kwargs)

datafed.CLI._queryList(offset, count): List saved queries.

datafed.CLI._queryView(qry_id): View a saved query by ID.

datafed.CLI._queryCreate(title, coll_mode, coll, id, text, tag, schema, meta, meta_err, owner, creator, time_from, time_to, public, category, sort, sort_rev): Create a saved query.

datafed.CLI._queryUpdate(qry_id, title, coll_mode, coll, id, text, tag, schema, meta, meta_err, owner, creator, time_from, time_to, public, category, sort, sort_rev): Update a saved query. The title and search terms of a query may be updated; however, search scope cannot currently be changed. To remove a term, specify an empty string (“”) for the associated option.

datafed.CLI._queryDelete(qry_id): Delete a saved query by ID.

datafed.CLI._queryExec(qry_id, offset, count): Execute a saved query by ID.

datafed.CLI._queryRun(coll_mode, coll, id, text, tag, schema, meta, meta_err, owner, creator, time_from, time_to, public, category, sort, sort_rev, offset, count)

Run a direct query on data or collections. The default scope is the current authenticated user. If collections are specified, they must be in the same overall search scope. At least one search term must be specified.

The sort option ‘text’ is a text-matching relevance ranking and only works if a text term is specified in the query. The –sort-rev option does not work with text relevance matching. The sort options ‘ct’ and ‘ut’ are creation and update times, respectively.

datafed.CLI._user()

datafed.CLI._userListCollab(offset, count): List all users that are collaborators. Collaborators are defined as users that have projects in common with the current user, or that have data- sharing relationships with the current user.

datafed.CLI._userListAll(offset, count): List all users.

datafed.CLI._userView(uid): View user information.

datafed.CLI._userWho(): Show current authenticated user ID.

datafed.CLI._project()

datafed.CLI._projectList(owned, admin, member, offset, count): List projects associated with current user. List projects that are owned or managed by the current user, as well as projects were the current user is a member.

datafed.CLI._projectView(proj_id): View project information. Current user must have a role (owner, manager, or member) within the project specified by the ID argument.

datafed.CLI._shares(users, projects): List users and/or projects sharing data with current user.

datafed.CLI._task()

datafed.CLI._taskList(time_from, to, since, status, offset, count): List recent tasks. If no time or status filter options are provided, all tasks initiated by the current user are listed, most recent first. Note that the DataFed server periodically purges tasks history such that only up to 30 days of history are retained.

datafed.CLI._taskView(task_id): Show task information. Use the ID argument to view a specific task record, or omit to view the latest task initiated by the current user.

datafed.CLI._ep()

datafed.CLI._epGet(): Get Globus endpoint for the current session. At the start of a session, the current endpoint will be set to the default endpoint, if configured.

datafed.CLI._epSet(endpoint): Set endpoint for the current session. If no endpoint is given, the default endpoint will be set as the current endpoint, if configured.

datafed.CLI._epList(): List recently used endpoints.

datafed.CLI._epDefault()

datafed.CLI._epDefaultGet(): Show the default Globus endpoint.

datafed.CLI._epDefaultSet(current, endpoint): Set the default Globus endpoint. The default endpoint will be set from the ‘endpoint’ argument, or if the ‘–current’ options is specified, from the currently active endpoint.

datafed.CLI._setup(ctx): Setup local credentials. This command installs DataFed credentials for the current user in the configured client configuration directory. Subsequent use of the DataFed CLI will read these credentials instead of requiring manual authentication.

datafed.CLI._verbositySet(level): Set/display verbosity level. The verbosity level argument can be 0 (lowest), 1 (normal), or 2 (highest). If the the level is omitted, the current verbosity level is returned.

datafed.CLI._help_cli(ctx, command): Show DataFed CLI help. Include a command name as the argument to see command-specific help.

datafed.CLI._exit_cli(): Exit an interactive session. Ctrl-C may also be used to exit the shell.

datafed.CLI._print_msg(level, message, err=False)

datafed.CLI._print_ack_reply(reply=None)

datafed.CLI._print_listing(message)

datafed.CLI._print_user_listing(message)

datafed.CLI._print_proj_listing(message)

datafed.CLI._print_endpoints(message)

datafed.CLI._print_data(message)

datafed.CLI._print_batch(message)

datafed.CLI._print_coll(message)

datafed.CLI._print_deps(dr)

datafed.CLI._print_task_listing(message)

datafed.CLI._print_task(message)

datafed.CLI._print_task_array(message)

datafed.CLI._print_user(message)

datafed.CLI._print_proj(message)

datafed.CLI._print_path(message)

datafed.CLI._print_query(message)

datafed.CLI._wrap_text(text, prefix, indent, compact=False)

datafed.CLI._resolve_id(df_id)

datafed.CLI._resolve_coll_id(coll_id, context=None)

datafed.CLI._generic_reply_handler(reply, printFunc)

datafed.CLI._setWorkingCollectionTitle()

datafed.CLI._arrayToCSV(items, skip)

datafed.CLI._arrayToDotted(items, skip=0)

datafed.CLI._printJSON(json, cur_indent, indent)

datafed.CLI._printJSON_List(json, cur_indent, indent)

datafed.CLI._bar_adaptive_human_readable(current, total, width=80)

datafed.CLI._initialize(opts)

datafed.CLI._addConfigOptions()