Crate xvc_pipeline

Pipeline management commands and data structures

This contains CLI structs for xvc pipeline subcommands, init function to run during xvc init for pipeline related initialization, cmd_pipeline and [handle_step_cli] functions to dispatch the options to subcommands.

Modules

deps

Step dependencies implementation

error

Error and Result types for the pipelines crate

Structs

CommandProcess

Used for encapsulating a process and its outputs. This is used to associate steps with running processes.

PipelineCLI

Pipeline management commands

RunCLI

Run a pipeline

XvcPipeline

A pipeline is a collection of steps that are run in a specific order. This struct defines the name of it.

XvcPipelineRunDir

A pipeline run directory where the pipeline is run. It should be within the workspace to be portable across systems.

XvcPipelineSchema

Defines the user editable pipeline schema used in xvc pipeline export and xvc pipeline import commands.

XvcStep

A step (stage) in a pipeline.

XvcStepCommand

Command to run for an XvcStep.

XvcStepSchema

User editable pipeline step schema used in xvc pipeline export and xvc pipeline import commands.

Enums

PipelineSubCommand

Pipeline management subcommands

XvcDependency

Represents variety of dependencies Xvc supports. This is to unify all dependencies without dynamic dispatch and having compile time errors when we miss something about dependencies.

XvcMetricsFormat

Possible formats for recognized metrics formats. Metrics files are where the pipeline writes its output in a structured format. We can read these files and use them to generate reports

XvcOutput

Possible outputs for the pipeline.

XvcParamFormat

Parsable formats of a parameter file

Functions

cmd_dag

Entry point for xvc pipeline dag command. Create a graph of the pipeline and output it in the specified format.

cmd_delete

Entry point for xvc pipeline delete command. It deletes the pipeline with the given name. It is not possible to delete the default pipeline.

cmd_export

Entry point for xvc pipeline export command. Export the pipeline and all its steps to a file. The file format is determined by the format parameter. If file is None, prints to stdout. If name is None, uses the default pipeline name from the config. If format is None, uses the default format from [XvcSchemaSerializationFormat::default()]

cmd_import

Entry point for xvc pipeline import command. Reads a pipeline definition in JSON or YAML formats and creates/updates it. If name is None, uses the pipeline name from the file. If file is None, reads from stdin. If format is None, uses the file extension to determine the format. If overwrite is true, overwrites the pipeline if it already exists.

cmd_list

Entry point for xvc pipeline list command. Lists all pipelines and their run directories.

cmd_new

Entry point for xvc pipeline new command. It creates a new pipeline with the given name. If workdir is None, uses the default workdir.

cmd_pipeline

Run xvc pipeline command. This is the entry point for the pipeline subcommand. It dispatches to the subcommands using PipelineCLI argument.

cmd_run

Entry point for xvc pipeline run command.

cmd_step_dependency

Entry point for xvc pipeline step dependency command. Add a set of new dependencies to the given step in the pipeline.

cmd_step_new

Creates a new step

cmd_step_output

Entry point for xvc pipeline step output command.

cmd_step_show

Entry point for xvc pipeline step show command.

cmd_step_update

Entry point for xvc pipeline step update command. Updates the command and invalidation strategy (when to run) of the given step.

cmd_update

Entry point for xvc pipeline update command. Can rename the pipeline, change the working directory or set the pipeline as default.

init

Initialize pipeline stores and save them.