Expand description
json_field_fold transform (canonical id "json_field_fold", v1.0.0).
Content-aware, losslessly reversible structural compression for generic JSON data
(InputFormat::Json). Where json_minify only removes whitespace, this transform removes
the dominant source of token waste in data-JSON: repeated object keys. An array of N
objects that all share the same keys emits each key N times; folding rewrites it to a
columnar form that emits each key exactly once.
[ {"id":1,"role":"member"}, {"id":2,"role":"member"} ]
-> {"__tf_cols__":["id","role"],"__tf_rows__":[[1,"member"],[2,"member"]]}The fold is applied recursively (values are folded before their enclosing array is), so
nested arrays-of-objects fold too. It is a pure structural rewrite — every value is
preserved — so unfold_json reconstructs the original data exactly (as a serde_json
value; number spelling like 1e10 may be renormalized by re-serialization, but the
numeric value is unchanged). The pipeline gates adoption on that round-trip
(round_trips), so a fold that would ever lose data is rolled back rather than emitted.
Enums§
Constants§
- TRANSFORM_
ID - Canonical transform id, as registered with the pipeline.
- TRANSFORM_
VERSION - Semantic version of this transform’s output behavior.
Functions§
- fold_
json - Folds arrays of homogeneous objects in
inputinto columnar{__tf_cols__, __tf_rows__}form. No-op-safe: empty input returns empty; input with no foldable arrays returns a (compact) re-serialization with the same data. - round_
trips - True iff unfolding
afterreproducesbeforeexactly (as JSON values). The pipeline’s safety gate for this transform — folding is only adopted when this holds. - unfold_
json - Inverse of
fold_json: expands every{__tf_cols__, __tf_rows__}node back into an array of objects. This is the reversible half that makes the transform safe.