Expand description
Pluggable state store for incremental replication bookmarks.
Sources that support incremental replication need to remember where they
left off across runs. This module defines the StateStore trait that
the pipeline orchestrator uses to read and persist that progress, plus two
ready-to-use implementations:
MemoryStateStore— in-process map. Useful for tests and for runs that intentionally start fresh each time.FileStateStore— one JSON file per key, written via atomic rename so a crash mid-update never leaves the bookmark torn.
Heavier backends (Redis, PostgreSQL) live in their own crates so
faucet-core stays dependency-light. They implement the same trait.
Structs§
- File
State Store - File-backed
StateStore. Each key maps to a JSON file at{root}/{safe_filename(key)}.json, written via atomic rename. The filename stem percent-encodes:as%3Aso keys using thepipeline:rest:issuesconvention are valid on Windows. - Memory
State Store - In-memory
StateStorefor tests and ephemeral pipelines.
Constants§
- DOCTOR_
SENTINEL_ KEY - Sentinel key used by state-store
check()probes. Valid pervalidate_state_keyand deleted after the probe so it leaves no residue.
Traits§
- State
Store - Persistent key/value store for replication bookmarks and pipeline checkpoints.
Functions§
- validate_
state_ key - Reject keys that could escape the storage namespace or break filename
rules on common filesystems. Allowed: ASCII letters, digits,
_,-,:,.. Empty keys are rejected.