#[non_exhaustive]pub struct QualityPolicy { /* private fields */ }Expand description
What a report must satisfy.
Build one with the chainable setters, then evaluate it
against a report. Requirements are evaluated in a fixed order that does not
depend on the order they were added, so two callers stating the same policy
get the same document out.
Implementations§
Source§impl QualityPolicy
impl QualityPolicy
Sourcepub fn new() -> Self
pub fn new() -> Self
An empty policy. It states no requirement, so evaluating it as-is is a
PolicyError::NoRequirements.
Sourcepub fn min_rows(self, min: usize) -> Self
pub fn min_rows(self, min: usize) -> Self
Require the scan to have read at least min rows, so an empty or
truncated extract fails instead of passing every other check.
Sourcepub fn min_quality_score(self, min: f64) -> Self
pub fn min_quality_score(self, min: f64) -> Self
Require the overall quality score to be at least min (0..=100).
Sourcepub fn min_dimension_score(self, dimension: QualityDimension, min: f64) -> Self
pub fn min_dimension_score(self, dimension: QualityDimension, min: f64) -> Self
Require one dimension’s score to be at least min (0..=100).
Sourcepub fn max_null_percentage(self, column: impl Into<String>, max: f64) -> Self
pub fn max_null_percentage(self, column: impl Into<String>, max: f64) -> Self
Require the named column’s null percentage to be at most max
(0..=100). Takes precedence over
max_null_percentage_any for that
column.
Sourcepub fn max_null_percentage_any(self, max: f64) -> Self
pub fn max_null_percentage_any(self, max: f64) -> Self
Require every other profiled column’s null percentage to be at most
max (0..=100).
Sourcepub fn max_duplicate_rows(self, max: usize) -> Self
pub fn max_duplicate_rows(self, max: usize) -> Self
Require the report to count at most max duplicate rows.
Sourcepub fn require_quality(self) -> Self
pub fn require_quality(self) -> Self
Require the report to carry a quality assessment at all.
Without this, a report with no assessment leaves quality requirements unevaluated and the verdict inconclusive. With it, the absence is itself a violation — which is what a pipeline wants when quality is supposed to be configured on.
Sourcepub fn require_dimension(self, dimension: QualityDimension) -> Self
pub fn require_dimension(self, dimension: QualityDimension) -> Self
Require a specific dimension to have been assessed.
Sourcepub fn max_quality_score_drop(self, points: f64) -> Self
pub fn max_quality_score_drop(self, points: f64) -> Self
Allow the overall quality score to drop at most points (0..=100)
below the baseline’s. Needs evaluate_against.
Sourcepub fn max_dimension_score_drop(
self,
dimension: QualityDimension,
points: f64,
) -> Self
pub fn max_dimension_score_drop( self, dimension: QualityDimension, points: f64, ) -> Self
Allow one dimension’s score to drop at most points (0..=100) below
the baseline’s. Needs evaluate_against.
Sourcepub fn max_null_percentage_increase(
self,
column: impl Into<String>,
points: f64,
) -> Self
pub fn max_null_percentage_increase( self, column: impl Into<String>, points: f64, ) -> Self
Allow the named column’s null percentage to rise at most points
(0..=100) above the baseline’s. Takes precedence over
max_null_percentage_increase_any
for that column. Needs evaluate_against.
Sourcepub fn max_null_percentage_increase_any(self, points: f64) -> Self
pub fn max_null_percentage_increase_any(self, points: f64) -> Self
Allow every other column both reports profiled to have its null
percentage rise at most points (0..=100) above the baseline’s. A
column only the evaluated report has is not covered: it has no
baseline, and gets a rule only when the policy names it. Needs
evaluate_against.
Sourcepub fn require_columns<I, S>(self, columns: I) -> Self
pub fn require_columns<I, S>(self, columns: I) -> Self
Require each named column to be profiled. A column missing from the report fails only when the report shows its source lacks it: the run’s column selection included the column, and the column set covers the whole source (see the module docs).
Sourcepub fn forbid_removed_columns(self) -> Self
pub fn forbid_removed_columns(self) -> Self
Require every column the baseline profiled to be profiled still. The
rule is about the columns the baseline recorded: a column its source
held but it did not profile (a projection, or a key past a partial
read) is not one it can require. Needs
evaluate_against.
Sourcepub fn forbid_added_columns(self) -> Self
pub fn forbid_added_columns(self) -> Self
Require every column the report profiled to be one the baseline
profiled: a closed schema. Under PolicyScope::FullSource, a report
whose profiled columns may not be all of its source’s also gets one
check with no column, left unevaluated: a column selection that left
columns out, one that was not recorded, or a column set covering part
of the source. Needs evaluate_against.
Sourcepub fn scope(self, scope: PolicyScope) -> Self
pub fn scope(self, scope: PolicyScope) -> Self
Set what the requirements are statements about. Defaults to
PolicyScope::FullSource.
Sourcepub fn validate(&self) -> Result<(), PolicyError>
pub fn validate(&self) -> Result<(), PolicyError>
Check that the policy is evaluable, without needing a report.
Callers that read a policy out of configuration should do this at startup rather than discovering a bad threshold on the first dataset.
Sourcepub fn evaluate(
&self,
report: &ProfileReport,
) -> Result<GateResult, PolicyError>
pub fn evaluate( &self, report: &ProfileReport, ) -> Result<GateResult, PolicyError>
Evaluate the policy against a report.
Returns the structured outcome. The report is not modified, nothing is
printed, and the process is not exited: what to do about a failing gate
is the caller’s decision. A policy with a rule about change is a
PolicyError::BaselineRequired here; use
evaluate_against.
Sourcepub fn evaluate_against(
&self,
report: &ProfileReport,
baseline: &ReportSide,
) -> Result<GateResult, PolicyError>
pub fn evaluate_against( &self, report: &ProfileReport, baseline: &ReportSide, ) -> Result<GateResult, PolicyError>
Evaluate the policy against a report and a baseline it is compared with.
Every requirement is evaluated as evaluate does,
then each rule about change, in a fixed order: the overall score, the
dimensions in QualityDimension::all order, then the named columns
in name order and every other shared column in the baseline’s order.
The baseline may come from a flat summary, which leaves what it did
not record unrecorded. See the module docs for when a change
is decided.