Skip to main content

Module masking

Module masking 

Source
Available on crate feature masking only.
Expand description

PII detection + column-level masking policies (issue #206): a declarative, destination-scoped policy that classifies sensitive fields — by field-name pattern, by value detector (email / card-with-Luhn / SSN / phone / IPv4), or by explicit field list — and rewrites them per action (redact / hash / tokenize / partial).

Ordering. The masking pass runs first in the page loop — before the quality, contract, and schema-drift passes and before every sink write. So every downstream consumer (those passes, the sink, the DLQ, and the sink-side lineage sample) observes only masked values; PII never leaks to a sink, a dead-letter queue, or a lineage facet. Masking is value-only and key-preserving, and it keeps determinism (equal inputs → equal masked outputs), non-null-ness, and per-key uniqueness, so the later quality / contract checks stay meaningful over masked data.

Masking never fails a run or quarantines a record: matching fields are rewritten in place. The pipeline wires metrics in instrumented_apply_masking.

Modules§

compile
Compilation + fail-fast validation for the masking layer (issue #206). Turns a MaskingSpec into a CompiledMasking with pre-compiled regexes and a resolved Hasher, optionally scoped to a single destination sink. Bad regexes / empty rule sets surface as FaucetError::Config at config-load time, never mid-run.
config
Config-shaped types for the PII detection + column-masking layer (issue #206). Pure declarations — validation and compilation live in compile.rs, evaluation in the module root.

Structs§

CompiledMasking
A masking policy compiled and ready to apply to pages.
MaskHit
One field-masking event, for metrics/observability.
MaskRule
One masking rule: a matcher + the action to apply to matching fields, optionally scoped to a subset of destination sinks.
MaskingOutcome
Result of masking one page.
MaskingSpec
The masking: config block: a declarative, destination-scoped policy that classifies sensitive fields (by field-name pattern, by value detector, or by explicit field list) and applies a masking action per page. The masking pass runs first — before the quality, contract, and schema-drift passes and before every sink write — so PII never reaches a sink, the DLQ, or a lineage sample unmasked.
MatchSpec
How a rule classifies a field as sensitive. Any combination is allowed; a field matches the rule if any configured criterion matches (name pattern OR value detector OR explicit field name).

Enums§

Detector
Built-in value detectors. All are conservative (anchored full-string matches; the card detector additionally requires a valid Luhn checksum) so false positives stay rare — masking silently rewrites data, so a false positive is a data-quality bug, not just noise.
MaskAction
What to do with a value once a rule matches it.

Functions§

apply_masking
Apply the masking policy to one page. Pure: no metrics, no I/O, infallible.