Module core

term_guard

Module core 

Source
Expand description

Core validation types for the Term data quality library.

This module provides the fundamental types for defining and executing data validation suites, including checks, constraints, and results.

§Overview

The core module contains the essential building blocks for data validation:

  • ValidationSuite: A collection of checks to run against your data
  • Check: A named group of related constraints with a severity level
  • Constraint: Individual validation rules (implemented in the constraints module)
  • Level: Severity levels for checks (Error, Warning, Info)
  • ValidationResult: Results from running a validation suite

§Architecture

ValidationSuite
    ├── Check (Level: Error)
    │   ├── Constraint 1
    │   └── Constraint 2
    └── Check (Level: Warning)
        ├── Constraint 3
        └── Constraint 4

§Example

use term_guard::core::{ValidationSuite, Check, Level, ValidationResult};
use term_guard::core::builder_extensions::{CompletenessOptions, StatisticalOptions};
use term_guard::constraints::{Assertion, FormatType, FormatOptions};
use datafusion::prelude::*;

// Build a validation suite using the unified API
let suite = ValidationSuite::builder("customer_validation")
    .description("Validate customer data quality")
    .check(
        Check::builder("critical_fields")
            .level(Level::Error)
            .description("Critical fields must be valid")
            // Unified API for completeness
            .completeness("customer_id", CompletenessOptions::full().into_constraint_options())
            .completeness("email", CompletenessOptions::threshold(0.99).into_constraint_options())
            // Convenience method for primary key validation
            .primary_key(vec!["customer_id"])
            .build()
    )
    .check(
        Check::builder("data_quality")
            .level(Level::Warning)
            // Format validation API
            .has_format("email", FormatType::Email, 0.95, FormatOptions::default())
            // Combined statistics in one query
            .statistics(
                "age",
                StatisticalOptions::new()
                    .min(Assertion::GreaterThanOrEqual(18.0))
                    .max(Assertion::LessThan(120.0))
            )?
            .build()
    )
    .build();

// Create context and register data
let ctx = SessionContext::new();
// ... register your customer table ...

// Run validation
let results = suite.run(&ctx).await?;

// Process results
match results {
    ValidationResult::Success { report, .. } => {
        println!("All validations passed!");
        for issue in &report.issues {
            if issue.level == Level::Warning {
                println!("Warning: {}", issue.message);
            }
        }
    }
    ValidationResult::Failure { report } => {
        println!("Validation failures:");
        for issue in &report.issues {
            if issue.level == Level::Error {
                println!("Error in '{}': {}", issue.check_name, issue.message);
            }
        }
    }
}

§Constraint Status

Each constraint evaluation returns a status:

  • Success: The constraint passed
  • Failure: The constraint failed
  • Skipped: The constraint was skipped (e.g., no data)

§Performance Considerations

  • Constraints within a check may be optimized to run in a single query
  • Use the with_optimizer(true) option on ValidationSuite for best performance
  • Group related constraints in the same Check when possible

Re-exports§

pub use validation_context::current_validation_context;
pub use validation_context::ValidationContext;
pub use validation_context::CURRENT_CONTEXT;

Modules§

builder_extensions
Extended builder API for unified constraints.
validation_context
Validation context for passing runtime information to constraints.

Structs§

CacheStats
Cache statistics for monitoring.
Check
A validation check containing one or more constraints.
CheckBuilder
Builder for constructing Check instances.
ConstraintMetadata
Metadata associated with a constraint.
ConstraintOptions
Common options for constraint configuration.
ConstraintResult
The result of evaluating a constraint.
DebugContext
Debug context for validation execution.
DebugInfo
Debug information collected during validation execution.
DebugSummary
Summary of debug information.
ErrorReport
Detailed error report for failed validations.
LogicalResult
Result of evaluating a logical expression.
MultiSourceValidator
Multi-source validation engine for cross-table data validation.
MultiTableCheck
Fluent builder for multi-table validation checks.
TermContext
A managed DataFusion context for Term validation operations.
TermContextConfig
Configuration for creating a TermContext.
UnifiedCompletenessBase
Base implementation for unified completeness-style constraints.
ValidationIssue
A detailed validation issue found during checks.
ValidationMetrics
Metrics collected during validation.
ValidationReport
A validation report containing all issues found.
ValidationSuite
A collection of validation checks to be run together.
ValidationSuiteBuilder
Builder for constructing ValidationSuite instances.

Enums§

ColumnSpec
Specification for columns in a constraint.
ConstraintStatus
The status of a constraint evaluation.
DebugLevel
Debug level for validation execution.
Level
The severity level of a validation check.
LogicalOperator
Logical operators for combining multiple boolean results.
ValidationResult
The result of running a validation suite.

Traits§

CheckMultiTableExt
Extension trait for Check to provide fluent multi-table methods.
Constraint
A validation constraint that can be evaluated against data.
ConstraintOptionsBuilder
Builder pattern for constraint options.
UnifiedConstraint
Base trait for unified constraints.
ValidationResultDebugExt
Extension trait for ValidationResult to add debug information.