Utils

Helper and utility functions for the library.

Data:

`CONFIGURATION_FILE`	The expected configuration filename.
`EDITORS`	The list of possible editor exec names to test for getting a valid text editor.
`TIMESTAMP_FORMAT`	The datetime format string used for timestamps in experiment run reference names.

Functions:

`check_git_dirty_workingdir`()	Checks if git working directory is dirty or not.
`get_command_output`(cmd[, silent])	Runs the command passed and returns the full string output of the command (minus the final newline).
`get_conda_env`()	Returns printed output from running `conda env export --from-history` command.
`get_conda_list`()	Returns printed output from running `conda env export --from-history` command.
`get_configuration`()	Load the configuration file if available, with defaults for any keys not found.
`get_current_commit`()	Returns printed output from running `git rev-parse HEAD` command.
`get_editor`()	Returns a text editor to use for richer text entry, such as in providing experiment notes.
`get_os`()	Get the current OS name and version.
`get_pip_freeze`()	Returns printed output from running `pip freeze` command.
`get_py_opening_comment`(lines)	Parse the passed lines of a python script file for a top of file docstring.
`human_readable_mem_usage`(byte_count)	Takes the given byte count and returns a nicely formatted string that includes the suffix (K/M/GB).
`human_readable_time`(seconds)	Takes the given time in seconds and returns a nicely formatted string that includes the suffix.
`init_logging`([log_path, level, log_errors, …])	Sets up logging configuration, including the associated file output.
`run_command`(cmd)	Prints output from running a command as it occurs.
`set_logging_prefix`(prefix)	Set the prefix content of the logger, which is incorporated in the log formatter.

curifactory.utils.CONFIGURATION_FILE = 'curifactory_config.json': The expected configuration filename.

curifactory.utils.EDITORS = ['vim', 'nvim', 'emacs', 'nano', 'vi']: The list of possible editor exec names to test for getting a valid text editor.

curifactory.utils.TIMESTAMP_FORMAT = '%Y-%m-%d-T%H%M%S': The datetime format string used for timestamps in experiment run reference names.

curifactory.utils.check_git_dirty_workingdir() → str: Checks if git working directory is dirty or not. This is used to indicate potential reproducibility problems in the report and console output.

curifactory.utils.get_command_output(cmd, silent=False) → str

Runs the command passed and returns the full string output of the command (minus the final newline).

Parameters: cmd – Either a string command or array of strings, as one would pass to subprocess.run()

curifactory.utils.get_conda_env() → str: Returns printed output from running conda env export --from-history command.

curifactory.utils.get_conda_list() → str: Returns printed output from running conda env export --from-history command.

curifactory.utils.get_configuration() → dict

Load the configuration file if available, with defaults for any keys not found. The config file should be “curifactory_config.json” in the project root.

The defaults are:

{
    "experiments_module_name": "experiments",
    "params_module_name": "params",
    "manager_cache_path": "data/",
    "cache_path": "data/cache",
    "runs_path": "data/runs",
    "logs_path": "logs/",
    "notebooks_path": "notebooks/",
    "reports_path": "reports/",
    "report_css_path": "reports/style.css",
}

Returns: the dictionary of configuration keys/values.

curifactory.utils.get_current_commit() → str: Returns printed output from running git rev-parse HEAD command.

curifactory.utils.get_editor() → str: Returns a text editor to use for richer text entry, such as in providing experiment notes.

curifactory.utils.get_os() → str: Get the current OS name and version.

curifactory.utils.get_pip_freeze() → str: Returns printed output from running pip freeze command.

curifactory.utils.get_py_opening_comment(lines): Parse the passed lines of a python script file for a top of file docstring. This is used to get parameter and experiment file descriptions, so be sure to always comment your code!

curifactory.utils.human_readable_mem_usage(byte_count: int) → str

Takes the given byte count and returns a nicely formatted string that includes the suffix (K/M/GB).

Parameters: byte_count (int) – The number of bytes to convert into KB/MB/GB.

curifactory.utils.human_readable_time(seconds: float) → str

Takes the given time in seconds and returns a nicely formatted string that includes the suffix.

Parameters: seconds (float) – The time in seconds to convert.

curifactory.utils.init_logging(log_path=None, level=20, log_errors=False, include_process=False, no_color=False, quiet=False, plain=False)

Sets up logging configuration, including the associated file output.

Parameters

log_path (str) – Folder to store output logs in. If None, only log to console.
level – The logging level to output.
log_errors (bool) – Whether to include error messages in the log output or not.
include_process (bool) – Whether to include the PID prefix value in the logger. This is mostly only used for when the --parallel flag is used, to help track which log message is from which process.
no_color (bool) – Suppress colors in console output.
quiet (bool) – Suppress all console log output.
plain (bool) – Output plain text log rather than rich output.

curifactory.utils.run_command(cmd)

Prints output from running a command as it occurs.

Parameters: cmd – Either a string command or array of strings, as one would pass to subprocess.run()

curifactory.utils.set_logging_prefix(prefix): Set the prefix content of the logger, which is incorporated in the log formatter. Currently this is just used to include pid number when running in parallel.