Expand description
Pipeline topology, resource graph, and core traits for the ECL pipeline runner.
This crate defines:
- The resolved pipeline topology (
PipelineTopology,ResolvedStage) - Resource graph computation and parallel schedule derivation
- Core traits (
SourceAdapter,Stage) and their supporting types (PipelineItem,SourceItem,ExtractedDocument,StageContext)
The topology is computed from a PipelineSpec (from ecl-pipeline-spec)
at init time and is immutable during execution.
Re-exports§
pub use error::ResolveError;pub use error::ResolveResult;pub use error::SourceError;pub use error::StageError;pub use traits::ExtractedDocument;pub use traits::PipelineItem;pub use traits::PushSourceAdapter;pub use traits::Record;pub use traits::SourceAdapter;pub use traits::SourceItem;pub use traits::Stage;pub use traits::StageContext;
Modules§
- error
- Error types for the topology layer.
- resolve
- Topology resolution: converting a PipelineSpec into a PipelineTopology.
- resource_
graph - Resource graph: stages connected by shared resource declarations. Used to compute the parallel execution schedule.
- schedule
- Schedule computation: topological sort and resource-conflict layer grouping.
- traits
- Core pipeline traits: SourceAdapter, Stage, and supporting types.
Structs§
- Condition
Expr - A condition expression that determines whether a stage should run. Currently a simple string wrapper; the evaluator is deferred to a future milestone.
- Pipeline
Topology - The resolved pipeline, ready to execute.
Computed from
PipelineSpecat init time. Immutable during execution. - Resolved
Stage - A resolved stage: the concrete implementation with merged configuration.
- Retry
Policy - Retry policy with resolved, concrete values.
Unlike
RetrySpec(from ecl-pipeline-spec) which stores milliseconds asu64, this storesDurationvalues — the resolved form ready for use by the runner.