Crate taquba_bulk 

Bulk multi-step processing for Taquba.

This crate re-exports taquba_workflow::bulk, where the implementation lives. Depend on taquba-workflow and use its bulk module directly; this crate receives no further development.

Structs

Batch

A handle on one batch of a Bulk runner. Obtained from Bulk::batch or Bulk::new_batch.

BatchStatus

The durable state of a batch, read from its manifest and item markers by Batch::status. An item with no recorded outcome is neither succeeded nor failed: it has not run to a settlement of its own, or it was cancelled.

Bulk

Runs one Pipeline over many inputs in a single process. Build it with Bulk::builder, spawn the worker, then run batches through Bulk::batch, Bulk::new_batch or Bulk::run; several batches can run concurrently on one worker. One runner per process: taquba is single-writer.

BulkBuilder

Builder for a Bulk runner. Construct via Bulk::builder.

BulkCtx

Per-item execution context handed to Pipeline::run.

BulkReport

The outcome of a finished (or drained) bulk run, returned by Batch::run.

CostReport

A collection of named cost counters, accumulated as a pipeline runs and rolled up across a batch.

EffectsHandle

Application KV effects staged during a step, applied in the same transaction as the settlement that commits the step’s outcome.

JsonlSink

An OutputSink that writes one JSON object per line to an underlying writer. Each line holds key, status and either output (for succeeded items) or error (when one is present). Writes are serialized through a mutex so the sink can be shared across worker tasks.

NullSink

An OutputSink that discards every record. The default sink, for runs whose pipeline produces its results as side effects (writing to a database, calling an API) rather than through the output stream.

OutputRecord

One result record handed to an OutputSink as an item completes.

ProgressSnapshot

A point-in-time view of a batch run’s progress. Returned by Batch::progress and suitable for a status line or a polling UI.

StepError

Failure outcomes the runner can return.

Enums

StepErrorKind

Whether a StepError should retry or fail the run.

Traits

OutputSink

The write side of a bulk run. Implementations receive one OutputRecord per item as it reaches a terminal state, possibly from many tasks concurrently, so write takes &self and must handle its own synchronization.

Pipeline

Defines a per-item processing pipeline. Each bulk run executes one Pipeline for every input item independently, materialised internally as a crate run.

Functions

read_jsonl

Decode a JSONL reader into an iterator of typed items. Each non-empty line is parsed as one T; blank lines are skipped. Decode errors are yielded as Err so the caller decides whether to stop or continue.