Store

Local ‘database’ of runs class.

Classes:

ManagerStore(manager_cache_path)

Manages the mini database of metadata on previous experiment runs.

class curifactory.store.ManagerStore(manager_cache_path)

Manages the mini database of metadata on previous experiment runs. This is how we keep track of experiment run numbers etc. A metadata block for each run is stored in the manager cache path under store.json.

Note that the metadata blocks we keep track of for each run follows the following example:

{
    "reference": "example_experiment_1_2021-06-15-T100003",
    "experiment_name": "example_experiment",
    "run_number": 1,
    "timestamp": "2021-06-15-T100003",
    "commit": "",
    "param_files": ["example_params"],
    "params": { "example_params": [ [ "test_params", "44b5e428e7165975a3e4f0d1674dbe5f" ] ] },
    "full_store": false,
    "status": "complete",
    "cli": "experiment example_experiment -p example_params",
    "hostname": "mycomputer",
    "notes": ""
}

Parameters:: manager_cache_path (str) – The path to the directory to keep the store.json.

Methods:

`add_run`(mngr)	Add a new metadata block to the store for the passed `ArtifactManager` instance.
`get_experiment_runs`(experiment_name)	Get all the runs associated with the specified experiment name from the database.
`get_run`(ref_name)	Get the metadata block for the run with the specified reference name.
`load`()	Load the current experiment database from `store.json` into `self.runs`.
`save`()	Save the current database in `self.runs` into the `store.json` file.
`update_run`(mngr)	Updates the metadata in the database for the run associated with the passed `ArtifactManager`.

Attributes:

`path`	The location to store the `store.json`.
`runs`	The list of metadata blocks for each run.

add_run(mngr)

Add a new metadata block to the store for the passed ArtifactManager instance.

Note that this automatically calls the save() function.

Parameters:: mngr (ArtifactManager) – The manager to grab run metadata from.
Returns:: The newly created dictionary (metadata block) for the current manager’s run.
Return type:: dict

get_experiment_runs(experiment_name)

Get all the runs associated with the specified experiment name from the database.

Parameters:: experiment_name (str) – The experiment name to get all run metadata for.
Returns:: A list of all dictionaries (metadata blocks) that have the requested experiment name.
Return type:: list[dict]

get_run(ref_name)

Get the metadata block for the run with the specified reference name.

Parameters:: ref_name (str) – The run reference name, following the [experiment_name]_[run_number]_[timestamp] format.
Returns:: A dictionary (metadata block) for the run with the requested reference name, and the index of the run within the total list of runs.
Return type:: tuple[dict, int]

load(): Load the current experiment database from store.json into self.runs.

path: The location to store the store.json.

runs: The list of metadata blocks for each run.

save(): Save the current database in self.runs into the store.json file.

update_run(mngr)

Updates the metadata in the database for the run associated with the passed ArtifactManager.

This is currently just used to update the status and include any error messages if relevant, when an experiment finishes running.

Note that this automatically calls the save() function.

Parameters:: mngr (ArtifactManager) – The manager to grab run metadata from.
Returns:: The updated dictionary (metadata block) for the run. It returns None if the experiment isn’t found in the database.
Return type:: dict