Procedure

Convenience class for grouping stages and automatically passing a record between them.

Exceptions:

NoArtifactManagerError

Classes:

Procedure(stages, manager, name, …)

A defined list of stages to run in sequence, creating a record to associate with them.

exception curifactory.procedure.NoArtifactManagerError

class curifactory.procedure.Procedure(stages: list, manager: Optional[curifactory.manager.ArtifactManager] = None, name: Optional[str] = None, previous_proc: Optional[curifactory.procedure.Procedure] = None, records: Optional[list] = None)

A defined list of stages to run in sequence, creating a record to associate with them.

For the stage list, specify only the names of the functions.

Example

@stage(...)
def data_stage(...):
    # ...

@stage(...)
def model_stage(...):
    # ...

proc = Procedure(
    [
        data_stage,
        model_stage
    ],
    mngr)

Parameters

list[Callable] (stages) – A list of function names that are wrapped in @stage or @aggregate decorators. Note that if using an aggregate state, it _must_ be the first one in the list.
manager (ArtifactManager) – The manager to associate this procedure and corresponding record with. If this is None, a manager will need to be passed when .run() is called.
name (str) – An optional name for the procedure. Currently unused, may eventually be put into logging or reporting.
previous_proc (Procedure) – If specified and this procedure begins with an aggregate stage, use the previous_proc.records list of records.
records (List[Record]) – If specified and this procedure begins with an aggregate stage, use this list of records. Note that previous_proc takes precedence over this argument.

Note

If a procedure begins with an aggregate stage and neither previous_proc nor records are specified, it will automatically grab all existing records from the artifact manager.

Note that you can predefine a procedure and “apply”/run it later, since the .run() function can take both a manager and records list directly.

Example

@stage(...)
def data_stage(...):
    # ...

@stage(...)
def model_stage(...):
    # ...

proc = Procedure(
    [
        data_stage,
        model_stage
    ])

def run(paramsets, manager):
    for paramset in paramsets:
        proc.run(paramset, manager=manager)

Methods:

run(param_set[, record, hide, manager, records])

Run this procedure with the passed parameter set.

run(param_set: curifactory.params.ExperimentParameters, record: Optional[curifactory.record.Record] = None, hide: bool = False, manager: Optional[curifactory.manager.ArtifactManager] = None, records: Optional[list] = None) → curifactory.record.Record 

Run this procedure with the passed parameter set. This allows easily running multiple parameter sets through the same set of stages and automatically getting a separate record for each.

Parameters

param_set (ExperimentParameters) – The parameteter set to put into the record created for this procedure.
record (Record) – If you have a specific record you want the procedure to use (e.g. if you’re chaining multiple procedures and already have an applicable record to use from the previous one), pass it here. If unspecified, a new record will automatically be created for the passed args and relevant artifact manager.
hide (bool) – If True, don’t add the created record to the artifact manager.
manager (ArtifactManager) – If a manager hasn’t been set yet on this procedure, do so now. An exception will be thrown if the manager has already been set on this procedure.
records (list[Record]) – If this procedure begins with an aggregate and the list of input records wasn’t set on init, do so now.

Returns

The returned output from the last stage in self.stages.