Skip to main content

Module quality

Module quality 

Source
Available on crate feature quality only.
Expand description

Data-quality checks: declarative per-record and per-batch assertions that quarantine violating records to the DLQ or abort the run. Pure evaluation; the pipeline wires the DLQ routing in run_stream.

See docs/superpowers/specs/2026-05-29-quality-checks-design.md.

Re-exports§

pub use compile::CompiledQuality;
pub use config::BatchCheck;
pub use config::CompareOp;
pub use config::JsonType;
pub use config::OnFailure;
pub use config::QualitySpec;
pub use config::RecordCheck;

Modules§

batch
Per-batch check evaluation over the survivor slice. Aggregate checks return a single pass/fail; unique returns the indices of the duplicate rows (row-attributable).
compile
Compile a QualitySpec into a CompiledQuality: parse paths/regexes/ schemas once, validate the on_failure subset and numeric bounds. Failures surface as FaucetError::Config.
config
Config-shaped types for the data-quality layer. Pure declarations — no evaluation logic (that lives in record.rs / batch.rs) and no compilation (that lives in compile.rs).
record
Per-record check evaluation. Pure functions over &Value. A check returns Ok(()) on pass or Err(message) on fail; the message is surfaced in the DLQ envelope or the abort error.

Structs§

CheckTally
Per-check counters + elapsed time, keyed by check name. Emitted as metrics by the observability wrapper.
QualityOutcome
Result of applying the quality pass to one page.
QuarantinedRecord
A record removed from the page by a quality check, destined for the DLQ.

Functions§

apply_quality
Apply the full per-page quality pass. Pure: no metrics, no DLQ I/O.