Module quality_gate

Expand description

Declarative quality gates over a ProfileReport.

A pipeline that wants to stop on a bad extract has to turn a report into a decision. Doing that by hand means re-deciding, at every call site, what an absent metric means, whether a sampled scan can answer a question about the whole source, and which number belongs in the log line. This module makes that decision data: a QualityPolicy states the requirements and QualityPolicy::evaluate returns a GateResult describing what was checked and what happened.

Evaluation never exits the process, prints, or mutates the report.

§Verdicts and evidence

The verdict is three-valued, because a decision and the evidence behind it are separate facts. A requirement about the full source cannot be passed by a truncated or sampled scan: nothing was observed that rules the failure out. It can still be failed by one, when the scan witnessed a violation that more scanning cannot retract. So:

Verdict::Fail — at least one requirement was conclusively violated.
Verdict::Inconclusive — nothing was violated, and at least one requirement could not be evaluated (a metric was not analyzed, a column was not profiled, the evidence does not reach as far as the requirement).
Verdict::Pass — every requirement was evaluated and met.

Fail wins over Inconclusive: a witnessed violation is a decision.

§Baseline-relative rules

QualityPolicy::evaluate_against also reads a saved baseline, through the ReportDelta between it and the report, and evaluates rules about change: how far the overall score or a dimension score may drop, and how far a column’s null percentage may rise, in points. A change is decided only when both sides measured the same thing the same way. Otherwise the rule is left unevaluated, naming what differs:

a value the baseline does not carry (NotEvaluated::NotInBaseline);
for a score, a different column set (NotEvaluated::SchemaChanged). A dataset-level score has the columns as its subject, so a column that appeared or vanished changes what is measured, not how good it is. A column’s own rules stay decisive, and a new column has no baseline rule until a policy names it;
for the overall score, a different set of assessed dimensions (NotEvaluated::DimensionsChanged), since it averages those;
a measurement definition the rule depends on that the two sides do not both record identically (NotEvaluated::DefinitionNotComparable);
a setting that moves the score on unchanged data, such as the score weights or a semantic hint, that differs or that either side did not record (NotEvaluated::SettingChanged, NotEvaluated::SettingUnrecorded);
under PolicyScope::FullSource, evidence on either side that does not cover the whole source.

A schema change from the baseline is reported as a Diagnostic in the result’s baseline.diagnostics, whatever the policy states: a column added, removed, retyped while holding values on both sides, or holding only nulls where the baseline had values. A diagnostic is a warning, never a fourth verdict, and does not move the verdict.

Rules about the column set decide on whether a column is there. QualityPolicy::require_columns names columns the report must have; QualityPolicy::forbid_removed_columns requires every baseline column, and QualityPolicy::forbid_added_columns allows no column the baseline lacked. A missing column fails such a rule only when the side without it shows its source lacks it: that side recorded its column selection, the selection included the column, and its column set covers the whole source. A source that declares its columns up front (a CSV header, a Parquet schema, a DataFrame, a query result) covers it however few rows were read; for JSON, JSONL and streams the columns are the keys of the records read, so only a scan of every record does. Otherwise the rule is unevaluated: NotEvaluated::SettingChanged or NotEvaluated::SettingUnrecorded on the setting columns, NotEvaluated::ColumnNotProfiled for a required column the run did not select, or NotEvaluated::EvidenceIncomplete.

§Example

use dataprof_runtime::{ProfileReport, QualityPolicy, Verdict};

fn gate(report: &ProfileReport) -> Result<bool, Box<dyn std::error::Error>> {
    let result = QualityPolicy::new()
        .min_quality_score(90.0)
        .max_null_percentage("customer_id", 0.0)
        .max_null_percentage_any(20.0)
        .require_quality()
        .evaluate(report)?;

    if result.verdict != Verdict::Pass {
        for violation in result.violations() {
            eprintln!("{}: {}", violation.code, violation.message);
        }
    }
    Ok(result.passed())
}

Structs§

BaselineComparison: The two reports a baseline-relative evaluation compared, and how their column sets differ.
BaselineReading: What the baseline said about the metric a rule about change reads.
Check: One requirement and what the report said about it.
CheckBounds: The interval a sampled score was decided on.
Diagnostic: A schema change between the baseline and the report.
GateResult: The structured result of evaluating a QualityPolicy.
QualityPolicy: What a report must satisfy.

Enums§

CheckCode: Stable identifier for the requirement a check came from.
CheckStatus: What happened to one requirement.
DiagnosticCode: Stable identifier for a kind of schema change.
Evidence: Whether the data behind a number covers the whole source.
EvidenceGap: Why the data behind a number falls short of the whole source.
Expectation: The constraint a check applied.
MetricValue: A number a check read, or a threshold it compared against.
NotEvaluated: Why a requirement was not evaluated.
PolicyError: A policy that cannot be evaluated as written.
PolicyScope: What the requirements in a policy are statements about.
RequiredMetric: A metric a policy requires to have been analyzed.
SettingDifference: How a setting that can produce a diagnostic’s change compares across the two reports. A diagnostic carrying one may not reflect a change in the data.
Verdict: The overall outcome of a policy.