Expand description
Config types for the top-level profiling: block (#708).
Structs§
- Profiling
Spec - Learned column profiles with drift detection. Every run profiles the records it wrote (null rate, distinct count, min / max / mean, string length, type mix, top values) into a bounded-memory sketch, compares the result against a rolling baseline of earlier successful runs, and reports statistically significant changes per column — no thresholds to write.
Enums§
- OnProfile
Drift - The action taken when a column’s profile drifts from its baseline.
Constants§
- DEFAULT_
CATEGORICAL_ MAX_ DISTINCT - Default distinct-count ceiling above which a column is treated as high-cardinality (no top values are published for it).
- DEFAULT_
MAX_ COLUMNS - Default cap on profiled columns per dataset.
- DEFAULT_
MIN_ HISTORY - Default number of baseline runs required before drift detection fires.
- DEFAULT_
NEW_ VALUE_ MIN_ SHARE - Default minimum share a previously unseen categorical value must reach before it is reported as a new value.
- DEFAULT_
PSI_ THRESHOLD - Default population-stability-index threshold for categorical drift.
- DEFAULT_
TOP_ VALUES - Default number of most-frequent values kept per categorical column.
- DEFAULT_
WINDOW - Default rolling-window size (successful runs kept as the baseline).
- MAX_
MAX_ COLUMNS - Hard ceiling on
max_columns(bounds per-run memory). - MAX_
TOP_ VALUES - Hard ceiling on
top_values(bounds the per-column sketch).