Curifactory

Usage

  • User Guide
    • Installation
      • Tab completion
    • Getting Started
      • Basic components
        • Artifact manager
        • Parameters
        • Records
        • Stages
      • Caching and reporting
        • Caching
        • Lazy loading
        • Reporting
      • Experiment organization
        • Basic mechanics
        • The experiment script file
        • The experiment CLI
        • Parameter files
        • Organization of growing projects
      • Next steps
    • Components
      • Component interactions
        • Stage flow
        • Procedure flow
        • Experiment flow
      • Anatomy of a stage
      • Stage parameters
      • Aggregate parameters
    • Parameter files and parameter sets
      • The ExperimentParameters class
      • Programmatic definition
      • Using parameters
      • Parameter set hashes and operational parameters
      • Subparameter classes
    • Cache
      • Using cachers
        • Cacher arguments
        • Inline cachers
      • Metadata
      • Lazy cache objects
        • Lazy resolve
    • Hashing Mechanics
      • Skipping parameters in a single set
    • Experiment Graph (DAG)
    • Reports
      • Report structure
      • Reportables
      • Reportable groups
    • Configuration and directory structure
      • Project directories
      • Configuration
    • CLI Guide
      • Running an experiment
      • Filtering to specific parameter sets
      • Caching controls
        • Specifying cache directory (-c, --cache)
        • Overwriting cached data (--overwrite, --overwrite-stage)
        • Full stores (--store-full, --dry-cache)
        • Lazy cache objects (--lazy, --ignore-lazy)
      • Parallel runs
      • Listing experiments and parameters
      • Experiment run notes
      • Hosting HTML reports
      • Getting Experiment DAG and Parameter Info
        • Experiment maps/cache status
        • Parameter set hashes/parameters
        • Cache paths
      • Full reference
        • Parameter names (-p, --params`)
        • Parameter set names (-n, --names)
        • Argument set indices (--indices)
        • Global arument set indices (--global-indices)
        • Parallel (--parallel)
        • Parallel mode (--parallel-safe)
        • Suppress logging (--no-log)
        • Suppress report generation (--no-report)
        • Overwrite cache (--overwrite)
        • Overwrite cache for stage (--overwrite-stage)
        • Full store (-s, --store-full)
        • Cache directory (-c, --cache)
        • Force lazy caching (--lazy)
        • Force no lazy caching (--ignore-lazy)
        • Include debug logs (-v, --verbose)
        • Progress bars (--progress)
        • Only print experiment map (--map)
        • Only print experiment hashes and names (--hashes)
        • Only print parameter set contents (--print-params)
        • Only print artifact cache paths (--paths)
        • Run experiment purely linearly instead with DAG (--no-dag)
        • Suppress console log output (--quiet)
        • Suppress terminal colors in case of unrecognized non-support (--no-color)
        • Suppress rich formatting of logging (--plain)
        • Suppress all file output (--dry)
        • Suppress writing to cache (--dry-cache)
        • Include errors in log files (--log-errors)
        • Custom cache name prefix (-n, --name)
        • Experiment run notes (--notes)
        • Export experiment in docker container (--docker)
        • Export experiment explorer as a jupyter notebook (--notebook)
        • Serve HTML reports on a specified port and host (--port, --host)
    • Example Experiment
    • Tips and tricks
      • Minimum code to run stages
      • Common dataset for multiple different argsets
      • Branching in experiments
      • Using the git commit hash from reports
      • Softlinking data directories
    • Curifactory Cookbook
      • Distributed torchrun via external stage
      • Cachers for Pytorch Lightning models
    • Migration Guide
      • 0.14.0 - 0.15.0
        • Change any Record.args references to Record.params
        • Classes subclassing curifactory.args.ExperimentArgs should subclass curifactory.params.ExperimentParameters
      • 0.13.0 - 0.14.0
        • Aggregate stages should now specify inputs in the decorator

API

  • API Reference
    • Caching
      • Cacheable
        • Cacheable.cache_paths
        • Cacheable.check()
        • Cacheable.collect_metadata()
        • Cacheable.extension
        • Cacheable.extra_metadata
        • Cacheable.get_dir()
        • Cacheable.get_path()
        • Cacheable.load()
        • Cacheable.load_metadata()
        • Cacheable.metadata
        • Cacheable.name
        • Cacheable.path_override
        • Cacheable.prefix
        • Cacheable.record
        • Cacheable.save()
        • Cacheable.save_metadata()
        • Cacheable.set_record()
        • Cacheable.stage
        • Cacheable.subdir
        • Cacheable.track
      • FileReferenceCacher
        • FileReferenceCacher.check()
        • FileReferenceCacher.load()
        • FileReferenceCacher.save()
      • JsonCacher
        • JsonCacher.load()
        • JsonCacher.save()
      • Lazy
        • Lazy.load()
      • PandasCacher
        • PandasCacher.load()
        • PandasCacher.save()
      • PandasCsvCacher
      • PandasJsonCacher
      • PathRef
        • PathRef.load()
        • PathRef.save()
      • PickleCacher
        • PickleCacher.load()
        • PickleCacher.save()
      • RawJupyterNotebookCacher
        • RawJupyterNotebookCacher.load()
        • RawJupyterNotebookCacher.save()
    • DAG
      • DAG
        • DAG.analyze()
        • DAG.artifacts
        • DAG.build_execution_tree_recursive()
        • DAG.build_execution_trees()
        • DAG.child_records()
        • DAG.determine_execution_list()
        • DAG.determine_execution_list_recursive()
        • DAG.execution_list
        • DAG.execution_trees
        • DAG.find_leaves()
        • DAG.get_record_string()
        • DAG.is_leaf()
        • DAG.is_output_used_anywhere()
        • DAG.print_experiment_map()
        • DAG.records
      • ExecutionNode
        • ExecutionNode.chain_rep()
        • ExecutionNode.dependencies
        • ExecutionNode.parent
        • ExecutionNode.string_rep()
    • Docker
      • build_docker()
    • Experiment
      • collect_parameter_sets()
      • list_experiments()
      • list_params()
      • regex_lister()
      • run_experiment()
    • Hashing
      • PARAMETERS_BLACKLIST
      • add_params_combo_hash()
      • compute_hash()
      • get_param_set_hash_values()
      • get_parameter_hash_value()
      • hash_param_set()
      • param_set_string_hash_representations()
      • set_hash_functions()
    • Manager
      • ArtifactManager
        • ArtifactManager.artifacts
        • ArtifactManager.cache_path
        • ArtifactManager.conda_env
        • ArtifactManager.config
        • ArtifactManager.current_stage_name
        • ArtifactManager.display_all_reportables()
        • ArtifactManager.display_group_reportables()
        • ArtifactManager.display_info()
        • ArtifactManager.display_record_reportables()
        • ArtifactManager.display_reportable()
        • ArtifactManager.display_stage_graph()
        • ArtifactManager.display_stage_reportables()
        • ArtifactManager.dry
        • ArtifactManager.dry_cache
        • ArtifactManager.error
        • ArtifactManager.error_thrown
        • ArtifactManager.experiment_name
        • ArtifactManager.experiment_run_number
        • ArtifactManager.generate_report()
        • ArtifactManager.get_all_param_sets()
        • ArtifactManager.get_artifact_filename()
        • ArtifactManager.get_artifact_path()
        • ArtifactManager.get_grouped_reportables()
        • ArtifactManager.get_reference_name()
        • ArtifactManager.get_reportable_groups()
        • ArtifactManager.get_run_output_path()
        • ArtifactManager.get_str_timestamp()
        • ArtifactManager.get_ungrouped_reportables()
        • ArtifactManager.git_commit_hash
        • ArtifactManager.git_workdir_dirty
        • ArtifactManager.hostname
        • ArtifactManager.ignore_lazy
        • ArtifactManager.interactive
        • ArtifactManager.lazy
        • ArtifactManager.live_report_path_generated
        • ArtifactManager.lock()
        • ArtifactManager.logs_path
        • ArtifactManager.manager_cache_path
        • ArtifactManager.map
        • ArtifactManager.map_mode
        • ArtifactManager.map_records()
        • ArtifactManager.mapped_artifacts
        • ArtifactManager.notebooks_path
        • ArtifactManager.notes
        • ArtifactManager.os
        • ArtifactManager.overwrite
        • ArtifactManager.overwrite_stages
        • ArtifactManager.parallel_lock
        • ArtifactManager.parallel_mode
        • ArtifactManager.param_file_param_sets
        • ArtifactManager.parameter_files
        • ArtifactManager.pip_freeze
        • ArtifactManager.prefix
        • ArtifactManager.records
        • ArtifactManager.report_css_path
        • ArtifactManager.reportables
        • ArtifactManager.reports_path
        • ArtifactManager.reproduction_line
        • ArtifactManager.run_info
        • ArtifactManager.run_line
        • ArtifactManager.run_timestamp
        • ArtifactManager.runs_path
        • ArtifactManager.stage_active
        • ArtifactManager.status
        • ArtifactManager.store()
        • ArtifactManager.store_full
        • ArtifactManager.stored
        • ArtifactManager.unlock()
        • ArtifactManager.update_map_progress()
        • ArtifactManager.write_run_env_output()
    • Params
      • ExperimentParameters
        • ExperimentParameters.hash
        • ExperimentParameters.hash_representations
        • ExperimentParameters.name
        • ExperimentParameters.overwrite
        • ExperimentParameters.params_hash()
    • Procedure
      • NoArtifactManagerError
      • Procedure
        • Procedure.run()
    • Project
      • add_completion_to_rc()
      • initialize_project()
      • main()
    • Record
      • ArtifactRepresentation
        • ArtifactRepresentation.html_safe()
      • CacheAwareDict
        • CacheAwareDict.resolve
      • MapArtifactRepresentation
      • Record
        • Record.combo_hash
        • Record.get_dir()
        • Record.get_hash()
        • Record.get_path()
        • Record.get_record_index()
        • Record.get_reference_name()
        • Record.input_records
        • Record.is_aggregate
        • Record.make_copy()
        • Record.manager
        • Record.output
        • Record.params
        • Record.report()
        • Record.set_aggregate()
        • Record.set_hash()
        • Record.stage_cachers
        • Record.stage_inputs
        • Record.stage_inputs_names
        • Record.stage_kwargs_keys
        • Record.stage_outputs
        • Record.stage_suppress_missing
        • Record.stages
        • Record.state
        • Record.state_artifact_reps
        • Record.store_tracked_paths()
        • Record.stored_paths
        • Record.unstored_tracked_paths
    • Reporting
      • DFReporter
        • DFReporter.html()
      • FigureReporter
        • FigureReporter.group
        • FigureReporter.html()
        • FigureReporter.kwargs
        • FigureReporter.name
        • FigureReporter.path
        • FigureReporter.qualified_name
        • FigureReporter.render()
        • FigureReporter.rendered
        • FigureReporter.stage
      • HTMLReporter
        • HTMLReporter.group
        • HTMLReporter.html()
        • HTMLReporter.html_string
        • HTMLReporter.name
        • HTMLReporter.path
        • HTMLReporter.qualified_name
        • HTMLReporter.rendered
        • HTMLReporter.stage
      • ImageReporter
        • ImageReporter.html()
        • ImageReporter.render()
      • JsonReporter
        • JsonReporter.html()
      • LatexTableReporter
        • LatexTableReporter.html()
      • LinePlotReporter
        • LinePlotReporter.html()
        • LinePlotReporter.render()
      • Reportable
        • Reportable.group
        • Reportable.html()
        • Reportable.name
        • Reportable.path
        • Reportable.qualified_name
        • Reportable.record
        • Reportable.render()
        • Reportable.rendered
        • Reportable.stage
      • map_full_svg()
      • map_single_svg()
      • prepare_report_path()
      • render_graph()
      • render_report_all_reportables()
      • render_report_argset_dump()
      • render_report_detailed_stage_maps()
      • render_report_head()
      • render_report_info_block()
      • render_report_reportables_toc()
      • render_report_stage_map()
      • render_report_toc()
      • render_reportable()
      • run_report()
      • update_report_index()
    • Staging
      • CachersMismatchError
      • EmptyCachersError
      • ExecutingWithSkippedInputError
      • InputSignatureError
      • OutputSignatureError
      • SkippedOutput
      • aggregate()
      • stage()
    • Store
      • ManagerStore
        • ManagerStore.add_run()
        • ManagerStore.get_experiment_runs()
        • ManagerStore.get_run()
        • ManagerStore.load()
        • ManagerStore.path
        • ManagerStore.runs
        • ManagerStore.save()
        • ManagerStore.update_run()
    • Utils
      • CONFIGURATION_FILE
      • EDITORS
      • PrefixedLogFactory
      • StreamToLogger
        • StreamToLogger.write()
      • TIMESTAMP_FORMAT
      • check_git_dirty_workingdir()
      • get_command_output()
      • get_conda_env()
      • get_conda_list()
      • get_configuration()
      • get_current_commit()
      • get_editor()
      • get_os()
      • get_pip_freeze()
      • get_py_opening_comment()
      • human_readable_mem_usage()
      • human_readable_time()
      • init_logging()
      • preview_object()
      • run_command()
      • set_logging_prefix()
    • Experimental Module
      • Artifact
        • Artifact
        • ArtifactFilter
        • ArtifactList
        • DBArtifact
        • StageReportables
        • pointer_based_property()
        • pointer_based_property_getter()
        • pointer_based_property_setter()
      • Experimental
        • convert_artifact_to_path()
        • get_artifact_paths()
        • get_manager()
        • get_output_path()
        • global_config()
        • report()
        • status()
      • Caching
        • AggregateArtifactCacher
        • Cacheable
        • DBCacher
        • DBTableCacher
        • FileReferenceCacher
        • JsonCacher
        • MetadataOnlyCacher
        • ParquetCacher
        • PathRef
        • PickleCacher
        • ReportablesCacher
        • TrackingDBTableCacher
      • CLI
        • completer_pipeline()
        • main()
      • Manager
        • CONFIGURATION_FILE
        • Manager
        • TIMESTAMP_FORMAT
      • Pipeline
        • Pipeline
        • PipelineFromRef
        • pipeline()
      • Reporting
        • HTMLReporter
        • Reportable
      • Stage
        • ConfigResolve
        • FunctionStub
        • OutputArtifactPathResolve
        • Stage
        • run()
        • stage()
      • Utils
        • AppendReplaceAction
        • FailsafeDict
        • PrefixedLogFactory
        • human_readable_mem_usage()
        • human_readable_time()
        • init_graphviz_graph()
        • run_command()
        • set_logging_prefix()
Curifactory
  • Search

© Copyright 2026, UT Battelle, LLC.

Built with Sphinx using a theme provided by Read the Docs.