taquba-workflow

Durable, at-least-once workflow runtime on top of the Taquba durable task queue.

taquba-workflow is the plumbing for any multi-step process that benefits from durable state between steps: idempotent step execution, retries with backoff, graceful restart, and terminal-state notifications. Implement StepRunner with bytes-in / bytes-out per-step logic; the runtime persists everything else.

Particularly well-suited for AI agent runs (see examples/rig_agent.rs for a Rig integration), but the runtime itself is framework-neutral and equally usable for ETL pipelines, document processing, payment flows, etc.

What this is / isn't

taquba-workflow is an imperative step orchestrator: at each step the runner decides what happens next via StepOutcome (Continue, Succeed, Fail). It is not:

A DAG executor. There's no declarative graph, no fan-out / fan-in, no dependency-driven scheduling.
An event-sourced workflow engine. There's no event-history replay, no per-side-effect recording.

Install

cargo add taquba-workflow taquba
cargo add tokio --features full

Enable the webhooks feature for WebhookTerminalHook:

cargo add taquba-workflow --features webhooks

Quick start

use std::sync::Arc;
use taquba::{Queue, object_store::memory::InMemory};
use taquba_workflow::{
    NoopTerminalHook, RunSpec, Step, StepError, StepOutcome, StepRunner, WorkflowRuntime,
};

struct EchoRunner;

impl StepRunner for EchoRunner {
    async fn run_step(&self, step: &Step) -> Result<StepOutcome, StepError> {
        Ok(StepOutcome::Succeed { result: step.payload.clone() })
    }
}

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let queue = Arc::new(Queue::open(Arc::new(InMemory::new()), "demo").await?);

    let runtime = WorkflowRuntime::builder(queue, EchoRunner, NoopTerminalHook).build();

    let worker = runtime.clone();
    tokio::spawn(async move { worker.run(std::future::pending::<()>()).await });

    let handle = runtime.submit(RunSpec {
        input: b"hello".to_vec(),
        ..Default::default()
    }).await?;
    println!("submitted run {}", handle.run_id);
    Ok(())
}

Examples

cargo run -p taquba-workflow --example single_step
cargo run -p taquba-workflow --example multi_step
ANTHROPIC_API_KEY=... cargo run -p taquba-workflow --example rig_agent
OPENAI_API_KEY=...    cargo run -p taquba-workflow --example rig_agent

rig_agent is a two-stage AI agent (research, then write) that demonstrates between-step durability: kill the process after step 0 and a fresh process resumes at step 1.

Step outcomes

Outcome	Effect
`StepOutcome::Continue { payload }`	Enqueue the next step immediately.
`StepOutcome::ContinueAfter { payload, delay }`	Schedule the next step `delay` from now.
`StepOutcome::Succeed { result }`	Ack; terminal hook fires `Succeeded`.
`StepOutcome::Fail { reason }`	Ack; terminal hook fires `Failed`. Runner verdict: no dead-letter.
`Err(StepError::transient(_))`	Retry per backoff up to `max_attempts`, then dead-letter.
`Err(StepError::permanent(_))`	Dead-letter immediately.

StepOutcome::Fail vs Err(StepError::permanent): a runner verdict acks normally; an infrastructure error dead-letters so operators can find it via queue.dead_jobs().

Reserved headers

Step jobs reserve the workflow.* prefix; submission rejects user headers starting with it. Other headers on RunSpec::headers thread through every step and reach the terminal hook on RunOutcome::headers.

Key	Meaning
`workflow.run_id`	Run identifier.
`workflow.step`	Zero-based step number.

Idempotency

Each step is enqueued with dedup_key = "run:{run_id}:{step_number}", preventing concurrent duplicate steps. But Taquba is at-least-once: a step can be claimed and executed twice if its lease expires before ack. StepRunner impls must be idempotent for the same (run_id, step_number).

Terminal hook

TerminalHook::on_termination fires once per run on Succeeded or Failed, receiving the submitter's headers and the runner's result or error. WebhookTerminalHook (behind the webhooks feature) fires HTTP callbacks via taquba-webhooks; set the per-run URL on RunSpec::headers["callback_url"].

License

Apache-2.0

taquba-workflow 0.1.0