Module catalog 

Available on crate feature catalog only.

Data Movement Catalog (#279) — the CLI-side write path.

The catalog accumulates, run over run, the operational history of every dataset a pipeline touches: identity, a deduplicated schema timeline, volume/freshness stats, and lineage edges. Storage rides the serve run-history backends (crate::serve::history); this module is the glue the executor calls after every successful root invocation:

• spec — the top-level catalog: config block (faucet schema catalog).
• model — pure URI canonicalization + sample-schema inference.
• CatalogHandle / connect_from_spec — the store handle carried on ExecuteOptions (serve passes its own history backend; the CLI runtimes connect from the catalog: block).
• record — the never-fails-the-run write (mirrors the SLA / lineage “log-and-continue” contract).

Gated on the catalog Cargo feature (implies serve for the storage backends and lineage for record sampling + column-lineage derivation).

Re-exports

pub use spec::CatalogSpec;

Modules

model

Pure helpers for building catalog observations (#279): dataset-URI canonicalization (fold ${now.*}-derived segments back to their tokens, redact credentials) and schema inference over the run’s record samples.

snapshot

Config-change preview (#374): build a redacted snapshot of the resolved + expanded config, and diff the current config against the last recorded one.

spec

Serde config types for the top-level catalog: block (#279).

Structs

CatalogHandle

The catalog store handle carried on ExecuteOptions. Cheaply cloneable.

Constants

DEFAULT_SAMPLE_RECORDS

Default per-side schema-inference sample cap when no catalog: block set one (the serve write path, which has no block).

Functions

connect_from_spec

Build a catalog store from the catalog: block. Errors are config-level (bad scheme / missing build feature) and fail fast at load time; an unreachable SQL backend degrades to in-memory via FallbackHistory (logged, run unaffected) exactly like faucet serve --history.

record

Persist one run’s catalog update. Monitoring must never take down the run it observes: any backend error is logged once per call and swallowed.

record_config_snapshot

Persist the latest resolved+expanded config snapshot for faucet plan --diff (#374). Best-effort, same never-fails-the-run contract as record.