pub struct ColumnProfile {
pub name: String,
pub data_type: DataType,
pub null_count: usize,
pub total_count: usize,
pub unique_count: Option<usize>,
pub unique_count_is_approximate: Option<bool>,
pub invalid_count: Option<usize>,
pub stats: ColumnStats,
pub patterns: Option<Vec<Pattern>>,
}Expand description
Profiling statistics for a single column.
Fields§
§name: String§data_type: DataType§null_count: usize§total_count: usize§unique_count: Option<usize>§unique_count_is_approximate: Option<bool>Whether unique_count is an approximate (HyperLogLog) estimate rather
than an exact distinct count.
None when unique_count is None (never computed); Some(false) for
an exact count; Some(true) once the cardinality estimator has spilled
to its HLL sketch (~1% relative error). Consumers running key or
high-cardinality/uniqueness gates must treat Some(true) as “do not rely
on this as an exact integer” – an exact-looking count with no provenance
is unsafe for those checks.
invalid_count: Option<usize>Non-null values that failed the column type’s raw validity predicate: non-finite or malformed numbers on numeric columns, and values that do not parse directly as calendar dates on date columns. The date predicate intentionally does not trim surrounding whitespace; descriptive date statistics may normalize whitespace independently.
For numeric columns, mean/std_dev cover
total_count - null_count - invalid_count values. For date columns,
this count audits the strict raw-value quality predicate. None means
the check did not run (another column type, or statistics skipped) —
never “no invalid values”, which is Some(0).
stats: ColumnStats§patterns: Option<Vec<Pattern>>Detected patterns, or None when pattern detection did not run.
None and Some(vec![]) are not interchangeable: the former means the
column was never scanned, the latter that it was scanned and nothing
matched. Consumers that gate on sensitivity – redaction, agent-facing
output – must treat None as “unknown”, never as “no sensitive data”.
Trait Implementations§
Source§impl Clone for ColumnProfile
impl Clone for ColumnProfile
Source§fn clone(&self) -> ColumnProfile
fn clone(&self) -> ColumnProfile
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source. Read more