Skip to main content

Module idempotency

Module idempotency 

Source
Expand description

Exactly-once / idempotent delivery primitives.

The pipeline issues a monotonic commit token for every page that carries a bookmark. The token is persisted in the StateStore value next to the bookmark and committed inside the sink’s own transaction, so a crash between “sink durably wrote” and “state persisted” is resolved on resume by skipping pages the sink already committed. See docs/superpowers/specs/2026-06-09-exactly-once-delivery-design.md.

Enums§

DeliveryMode
Delivery guarantee for a pipeline run.

Constants§

COMMIT_TOKEN_SCOPE_COL
Watermark column holding the pipeline state-key ({name}::{row_id}).
COMMIT_TOKEN_TABLE
Canonical watermark table the SQL sinks UPSERT the commit token into.
COMMIT_TOKEN_TOKEN_COL
Watermark column holding the latest committed token.
ICEBERG_SCOPE_PROP
Iceberg snapshot summary property names.
ICEBERG_TOKEN_PROP

Functions§

format_token
Render a page sequence as a fixed-width, lexicographically-ordered token.
parse_token
Parse a token produced by format_token. Returns None on garbage.
unwrap_state
Unwrap a stored state value into (bookmark, seq).
wrap_state
Wrap a bookmark + sequence into the exactly-once state value.