Stage

Classes:

ConfigResolve(config_key_name[, default])

Check global config at runtime for specified key, or use specified default if not found.

FunctionStub(name[, func_module])

OutputArtifactPathResolve(output_artifact_index)

Used for getting the path of an _output_ artifact as an _input_ to the stage.

Stage(function, args, kwargs, outputs[, ...])

Essentially a fancy "partial" that assigns outputs to instance attributes typed as Artifacts.

Functions:

run(output_names, *inputs)

stage(*outputs[, hashing_functions, pass_self])

class curifactory.experimental.stage.ConfigResolve(config_key_name, default=None)

Check global config at runtime for specified key, or use specified default if not found.

This can either be used in a stage def or cacher.

Methods:

resolve()

Parameters:

  • config_key_name (str)

  • default (Any)

resolve()
class curifactory.experimental.stage.FunctionStub(name, func_module=None)
class curifactory.experimental.stage.OutputArtifactPathResolve(output_artifact_index)

Used for getting the path of an _output_ artifact as an _input_ to the stage.

Methods:

resolve(stage)

resolve(stage)
class curifactory.experimental.stage.Stage(function, args, kwargs, outputs, hashing_functions=None, pass_self=False, db_id=None, dependencies=<factory>, computed=False, from_ref=False, hash_str=None, hash_debug=None)

Essentially a fancy “partial” that assigns outputs to instance attributes typed as Artifacts.

Attributes:

args

artifacts

computed

Mostly only used to help ensure output-less dependencies don't run more than once.

contextualized_name

db_id

dependencies

Explicit stage dependencies that don't have outputs used in this stage but are still required to run first.

from_ref

Indicates that it was populate by reference rather than by code - underlying function may not exist or be accessible if True

function

hash_debug

hash_str

hashing_functions

kwargs

name

outputs

pass_self

reportables

Methods:

compute_hash()

copy()

get_output_list()

Make it easier to act on outputs without always having to first chekc if it's a list or not.

get_parameter_value(param_index, param_name)

get_parameter_value_by_name(parameter_name)

hash_parameter(param_name, param_value)

load_from_uuid(uuid[, building_stages, ...])

Recurisvely load all input artifacts and their dependencies and then return the stage they feed into.

map([mapped, need, source])

reset_map()

resolve_arg([record_resolution, arg_index, ...])

resolve_args([record_resolution])

Handle any artifacts passed in as arguments.

resolve_template_string(str_to_format)

visualize([g])

Parameters:

  • function (callable)

  • args (list)

  • kwargs (dict)

  • outputs (list[Artifact] | Artifact)

  • hashing_functions (dict[str, callable])

  • pass_self (bool)

  • db_id (UUID)

  • dependencies (list[Stage])

  • computed (bool)

  • from_ref (bool)

  • hash_str (str)

  • hash_debug (dict[str, dict[str, Any]])

args: list
property artifacts
compute_hash()
Return type:

tuple[str, dict[str, dict[str, Any]]]

computed: bool = False

Mostly only used to help ensure output-less dependencies don’t run more than once.

property contextualized_name
copy()
db_id: UUID = None
dependencies: list[Stage]

Explicit stage dependencies that don’t have outputs used in this stage but are still required to run first. (Either set explicitly or use within a context manager)

from_ref: bool = False

Indicates that it was populate by reference rather than by code - underlying function may not exist or be accessible if True

function: callable
get_output_list()

Make it easier to act on outputs without always having to first chekc if it’s a list or not.

get_parameter_value(param_index, param_name)
get_parameter_value_by_name(parameter_name)
Parameters:

parameter_name (str)

hash_debug: dict[str, dict[str, Any]] = None
hash_parameter(param_name, param_value)
Return type:

tuple[str, any]

hash_str: str = None
hashing_functions: dict[str, callable] = None
kwargs: dict
static load_from_uuid(uuid, building_stages=None, building_artifacts=None, prepopulated_stage=None)

Recurisvely load all input artifacts and their dependencies and then return the stage they feed into. Note that function stubs are used by default.

Parameters:

  • building_stages (dict)

  • building_artifacts (dict)

map(mapped=None, need=True, source=None)
Parameters:

  • mapped (dict)

  • need (bool)

property name
outputs: list[Artifact] | Artifact
pass_self: bool = False
property reportables
reset_map()
resolve_arg(record_resolution=False, arg_index=None, arg_name=None)
Parameters:

  • record_resolution (bool)

  • arg_index (int)

  • arg_name (str)

resolve_args(record_resolution=False)

Handle any artifacts passed in as arguments.

Parameters:

record_resolution (bool)

Return type:

tuple[list, dict]

resolve_template_string(str_to_format)

  • {hash}

  • {stage_name}

  • {[PARAMETER_NAME]}

  • {[PARAMETER_NAME].name} - the name of the PARAMETER artifact (if it is in fact an artifact)

Parameters:

str_to_format (str)

Return type:

str

visualize(g=None, **kwargs)
curifactory.experimental.stage.run(output_names, *inputs)
Parameters:

output_names (str | list[str])

curifactory.experimental.stage.stage(*outputs, hashing_functions=None, pass_self=False)
Parameters:

  • outputs (list[Artifact])

  • hashing_functions (dict[str, callable])

  • pass_self (bool)