Skip to main content

QualityPolicy

Struct QualityPolicy 

Source
#[non_exhaustive]
pub struct QualityPolicy { /* private fields */ }
Expand description

What a report must satisfy.

Build one with the chainable setters, then evaluate it against a report. Requirements are evaluated in a fixed order that does not depend on the order they were added, so two callers stating the same policy get the same document out.

Implementations§

Source§

impl QualityPolicy

Source

pub fn new() -> Self

An empty policy. It states no requirement, so evaluating it as-is is a PolicyError::NoRequirements.

Source

pub fn min_rows(self, min: usize) -> Self

Require the scan to have read at least min rows, so an empty or truncated extract fails instead of passing every other check.

Source

pub fn min_quality_score(self, min: f64) -> Self

Require the overall quality score to be at least min (0..=100).

Source

pub fn min_dimension_score(self, dimension: QualityDimension, min: f64) -> Self

Require one dimension’s score to be at least min (0..=100).

Source

pub fn max_null_percentage(self, column: impl Into<String>, max: f64) -> Self

Require the named column’s null percentage to be at most max (0..=100). Takes precedence over max_null_percentage_any for that column.

Source

pub fn max_null_percentage_any(self, max: f64) -> Self

Require every other profiled column’s null percentage to be at most max (0..=100).

Source

pub fn max_null_count(self, column: impl Into<String>, max: usize) -> Self

Require the named column to hold at most max nulls, decided on the exact count. max_null_count(column, 0) states “no nulls” at any row count, where a zero null percentage is decided at the two decimals it is shown at. Takes precedence over max_null_count_any for that column.

Source

pub fn max_null_count_any(self, max: usize) -> Self

Require every other profiled column to hold at most max nulls.

Source

pub fn max_duplicate_rows(self, max: usize) -> Self

Require the report to count at most max duplicate rows.

Source

pub fn require_quality(self) -> Self

Require the report to carry a quality assessment at all.

Without this, a report with no assessment leaves quality requirements unevaluated and the verdict inconclusive. With it, the absence is itself a violation — which is what a pipeline wants when quality is supposed to be configured on.

Source

pub fn require_dimension(self, dimension: QualityDimension) -> Self

Require a specific dimension to have been assessed.

Source

pub fn max_quality_score_drop(self, points: f64) -> Self

Allow the overall quality score to drop at most points (0..=100) below the baseline’s. Needs evaluate_against.

Source

pub fn max_dimension_score_drop( self, dimension: QualityDimension, points: f64, ) -> Self

Allow one dimension’s score to drop at most points (0..=100) below the baseline’s. Needs evaluate_against.

Source

pub fn max_null_percentage_increase( self, column: impl Into<String>, points: f64, ) -> Self

Allow the named column’s null percentage to rise at most points (0..=100) above the baseline’s. Takes precedence over max_null_percentage_increase_any for that column. Needs evaluate_against.

Source

pub fn max_null_percentage_increase_any(self, points: f64) -> Self

Allow every other column both reports profiled to have its null percentage rise at most points (0..=100) above the baseline’s. A column only the evaluated report has is not covered: it has no baseline, and gets a rule only when the policy names it. Needs evaluate_against.

Source

pub fn require_columns<I, S>(self, columns: I) -> Self
where I: IntoIterator<Item = S>, S: Into<String>,

Require each named column to be profiled. A column missing from the report fails only when the report shows its source lacks it: the run’s column selection included the column, and the column set covers the whole source (see the module docs).

Source

pub fn forbid_removed_columns(self) -> Self

Require every column the baseline profiled to be profiled still. The rule is about the columns the baseline recorded: a column its source held but it did not profile (a projection, or a key past a partial read) is not one it can require. Needs evaluate_against.

Source

pub fn forbid_added_columns(self) -> Self

Require every column the report profiled to be one the baseline profiled: a closed schema. Under PolicyScope::FullSource, a report whose profiled columns may not be all of its source’s also gets one check with no column, left unevaluated: a column selection that left columns out, one that was not recorded, or a column set covering part of the source. Needs evaluate_against.

Source

pub fn scope(self, scope: PolicyScope) -> Self

Set what the requirements are statements about. Defaults to PolicyScope::FullSource.

Source

pub fn validate(&self) -> Result<(), PolicyError>

Check that the policy is evaluable, without needing a report.

Callers that read a policy out of configuration should do this at startup rather than discovering a bad threshold on the first dataset.

Source

pub fn evaluate( &self, report: &ProfileReport, ) -> Result<GateResult, PolicyError>

Evaluate the policy against a report.

Returns the structured outcome. The report is not modified, nothing is printed, and the process is not exited: what to do about a failing gate is the caller’s decision. A policy with a rule about change is a PolicyError::BaselineRequired here; use evaluate_against.

Source

pub fn evaluate_against( &self, report: &ProfileReport, baseline: &ReportSide, ) -> Result<GateResult, PolicyError>

Evaluate the policy against a report and a baseline it is compared with.

Every requirement is evaluated as evaluate does, then each rule about change, in a fixed order: the overall score, the dimensions in QualityDimension::all order, then the named columns in name order and every other shared column in the baseline’s order. The baseline may come from a flat summary, which leaves what it did not record unrecorded. See the module docs for when a change is decided.

Trait Implementations§

Source§

impl Clone for QualityPolicy

Source§

fn clone(&self) -> Self

Returns a duplicate of the value. Read more
1.0.0 (const: unstable) · Source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
Source§

impl Debug for QualityPolicy

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more
Source§

impl Default for QualityPolicy

Source§

fn default() -> Self

Returns the “default value” for a type. Read more

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
where ST: ?Sized, DT: ?Sized,

Source§

impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
where ST: ?Sized, DT: ?Sized,

Source§

impl<T> CloneToUninit for T
where T: Clone,

Source§

unsafe fn clone_to_uninit(&self, dest: *mut u8)

🔬This is a nightly-only experimental API. (clone_to_uninit)
Performs copy-assignment from self to dest. Read more
Source§

impl<T> DynClone for T
where T: Clone,

Source§

fn __clone_box(&self, _: Private) -> *mut ()

Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> Read<Exclusive, BecauseExclusive> for T
where T: ?Sized,

Source§

impl<T> ToOwned for T
where T: Clone,

Source§

type Owned = T

The resulting type after obtaining ownership.
Source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
Source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = !

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, !>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
Source§

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

Source§

fn vzip(self) -> V