Crate spate_coordination 

Distributed work coordination for Spate sources.

A leader-elected worker runs the source’s SplitPlanner to enumerate weighted work splits into a shared low-latency store, and publishes a desired assignment per instance; every worker leases the splits it was named for, heartbeats them, and cooperatively drains the ones it was not. Progress commits are epoch-fenced compare-and-swap writes on the durable split record: a fenced commit writes nothing, and committed progress can only replay, never regress (at-least-once — duplicates possible, loss never).

This crate implements the spate_core::coordination seam (re-exported here) over the public store::CoordinationStore trait:

• store::memory::MemoryStore — in-process, for tests and single-machine embedding.
• A NATS JetStream KV store (default nats feature, server >= 2.11) — the production backend.

Custom backends (Redis, etcd) implement the store trait; the protocol, fencing, election, and work assignment live above it and are shared.

Re-exports

pub use config::CoordinationConfig;

Modules

config

Coordinator configuration and its mechanical floors.

driver

Reusable source-side choreography for coordinated sources.

store

Store primitives the coordination protocol runs on.

Structs

ControlWaker

Wakes a coordinated source’s control-plane wait.

CoordinationError

Error from a coordination backend.

LeaseEpoch

Monotonic fencing token for one split’s lease, minted by the backend. Strictly increases across ownership changes of that split; a write presented under a superseded epoch is rejected.

PlanContext

What the backend hands the planner on each run.

PlannedSplit

A split as the planner submits it: the spec plus optional seed progress. Seeds are first-writer-wins — a seed never overwrites an existing record — which is the migration path for work that already has a pre-coordination checkpoint. A brand-new source plans without seeds.

SplitId

Deterministic identity of one split, minted by the planner.

SplitPlan

One planner run’s output.

SplitProgress

One split’s durable progress, written through the fenced commit and handed to the next owner in CoordinationEvent::Gained.

SplitSpec

One unit of leasable work, as the planner enumerated it.

StoreCoordinator

A SplitCoordinator over any CoordinationStore.

SystemClock

Real wall time — the production clock.

Enums

CoordinationErrorKind

Why a coordination operation failed.

CoordinationEvent

Ownership or job-state change surfaced by SplitCoordinator::poll.

PlanFinality

Whether a plan’s enumeration can still grow.

Constants

SPLIT_ID_MAX_LEN

Maximum length of a SplitId in bytes.

Traits

Clock

A monotonic clock. Returns tokio::time::Instant so the value slots straight into the existing lease/deadline fields without a type change.

SplitCoordinator

Per-worker coordination handle: leases splits toward a bounded working set, surfaces ownership changes, and owns the fenced progress commit.

SplitPlanner

Source-provided work enumerator, run only on the fleet’s elected leader — at job start and again on the replan interval while the plan is Open.

Type Aliases

MemoryCoordinator

StoreCoordinator over the in-memory store: tests and single-process embedding.

NatsCoordinator

StoreCoordinator over NATS JetStream KV: the production backend (server >= 2.11). Build the store with NatsStore::new — construction is synchronous; the connection is made lazily under the startup budget.

Sleep

A future that resolves when a Clock reaches some instant.