Skip to main content

Crate pipeline_toolkit

Crate pipeline_toolkit 

Source
Expand description

A small toolkit for building linear, async processing pipelines out of independently testable steps that communicate through a shared, type-keyed Context.

Each Step declares, via Step::access, exactly which types it reads, writes, or consumes from the Context. PipelineBuilder::build uses those declarations to catch wiring mistakes — a step reading a type nothing upstream produces — before the pipeline ever runs, and Context enforces them again at run time, so a step that reads or writes something it didn’t declare fails fast with PipelineError::AccessViolation instead of silently drifting from its declared contract.

§Examples

use pipeline_toolkit::{AccessSpec, Context, Flow, Pipeline, Step, StepError};

struct LoadName;

#[async_trait::async_trait]
impl Step<String> for LoadName {
    fn name(&self) -> &str {
        "load_name"
    }

    fn access(&self) -> AccessSpec {
        AccessSpec::new().writes::<String>()
    }

    async fn run(&self, ctx: &mut Context) -> Result<Flow<String>, StepError> {
        ctx.insert("Ada".to_string());
        Ok(Flow::Continue)
    }
}

struct Greet;

#[async_trait::async_trait]
impl Step<String> for Greet {
    fn name(&self) -> &str {
        "greet"
    }

    fn access(&self) -> AccessSpec {
        AccessSpec::new().reads::<String>()
    }

    async fn run(&self, ctx: &mut Context) -> Result<Flow<String>, StepError> {
        let name = ctx.get::<String>().expect("declared as read").clone();
        Ok(Flow::Done(format!("Hello, {name}!")))
    }
}

let pipeline = Pipeline::<(), String>::builder()
    .step(LoadName)
    .step(Greet)
    .build()?;

let greeting = block_on(pipeline.run(()))?;
assert_eq!(greeting, "Hello, Ada!");

Structs§

AccessSpec
The set of Context entry types a Step is allowed to read, write, or consume, keyed by TypeId.
AccessViolation
Records an undeclared Context access: a step read, wrote, or consumed a type its AccessSpec didn’t grant.
Arm
A named sequence of Steps that runs as one option of a Branch.
ArmBuilder
Incrementally assembles an Arm, returned by Arm::builder.
Branch
A framework-provided Step that runs one of several named arms, chosen at run time by a key-selecting condition function.
BranchBuilder
Incrementally assembles a Branch, returned by Branch::builder.
Context
A type-keyed store of at most one value per type, shared across the steps of a running Pipeline.
Loop
A framework-provided Step that runs the same Arm body repeatedly, do-while style, until a condition function says to stop or max_iterations is reached.
LoopBuilder
Incrementally assembles a Loop, returned by Loop::builder.
Pipeline
A validated, immutable sequence of Steps, built via Self::builder.
PipelineBuilder
Incrementally assembles a Pipeline, returned by Pipeline::builder.
StepError
The error type returned by Step::run.

Enums§

AccessKind
The kind of Context access a Step performed, used to describe an AccessViolation.
BuildError
Why PipelineBuilder::build rejected a pipeline.
Flow
What a Step wants the Pipeline to do next.
PipelineError
Why Pipeline::run failed.

Traits§

Step
A single unit of work in a Pipeline.