Skip to main content

Module state

Module state 

Source
Expand description

Pluggable state store for incremental replication bookmarks.

Sources that support incremental replication need to remember where they left off across runs. This module defines the StateStore trait that the pipeline orchestrator uses to read and persist that progress, plus two ready-to-use implementations:

  • MemoryStateStore — in-process map. Useful for tests and for runs that intentionally start fresh each time.
  • FileStateStore — one JSON file per key, written via atomic rename so a crash mid-update never leaves the bookmark torn.

Heavier backends (Redis, PostgreSQL) live in their own crates so faucet-core stays dependency-light. They implement the same trait.

Structs§

FileStateStore
File-backed StateStore. Each key maps to a JSON file at {root}/{safe_filename(key)}.json, written via atomic rename. The filename stem percent-encodes : as %3A so keys using the pipeline:rest:issues convention are valid on Windows.
MemoryStateStore
In-memory StateStore for tests and ephemeral pipelines.

Constants§

DOCTOR_SENTINEL_KEY
Sentinel key used by state-store check() probes. Valid per validate_state_key and deleted after the probe so it leaves no residue.

Traits§

StateStore
Persistent key/value store for replication bookmarks and pipeline checkpoints.

Functions§

validate_state_key
Reject keys that could escape the storage namespace or break filename rules on common filesystems. Allowed: ASCII letters, digits, _, -, :, .. Empty keys are rejected.