Module tabular_crush 

Deterministic tabular (CSV/TSV) crusher — columnar redundancy factoring for delimited data (#982, Headroom tabular-compressor port, GitLab Epic #973).

Real-world CSV/TSV dumps (DB exports, psql -A -F,, analytics extracts) are dominated by columns that repeat one value on every row (a status, region, tenant column) and by near-unique noise columns (UUIDs, timestamps). This module factors that out, mirroring crate::core::json_crush but for tables:

• Lossless: every constant column (exactly one distinct value across all data rows) is hoisted once into _const; the remaining columns are kept positionally in _rows, so a value never repeats more than it must. The transform is exactly reversible via reconstruct.
• Lossy: additionally drops near-unique high-entropy columns (recorded in _dropped); the exact original is recovered out-of-band via CCR, never from the text.

The compact form is serialized with serde_json (robust quoting, escapes and Unicode — no hand-rolled format that could desync the reader), and the output is a pure function of the input (columns walked in header order, _const keyed deterministically), so identical input yields byte-identical output (#498). The crusher never inflates: callers gate on the shared crate::core::json_crush::KEEP_DATA_DIVISOR threshold and a no-op input returns None.

Functions

crush_text_if_beneficial

Lossless columnar crush of delimited text, returning the compact JSON form only when it clears the beneficial reduction gate. None for non-tabular, ragged, or low-redundancy input — the caller keeps its own path.

crush_text_lossy_if_beneficial

Lossy columnar crush of text: drops near-unique high-entropy columns whose distinct-value ratio is >= drop_entropy. Returns the CrushResult only when a column was actually dropped (!lossless) AND the compact form at least halves the input (crate::core::json_crush::KEEP_DATA_DIVISOR). Because data is then lost, the caller MUST persist the verbatim original out-of-band (CCR) before emitting — the dropped columns are never reconstructible from the text. None for non-tabular, low-redundancy, or all-lossless input.

reconstruct

Rebuild the parsed rows ([header, ..data]) from crushed text. Exact for lossless forms; for lossy forms the _dropped columns are simply absent from the header and every row (recover them via CCR). None if text is not a tabular-crush document or is internally inconsistent.