Skip to main content

Module json_fold

Module json_fold 

Source
Expand description

json_field_fold transform (canonical id "json_field_fold", v1.0.0).

Content-aware, losslessly reversible structural compression for generic JSON data (InputFormat::Json). Where json_minify only removes whitespace, this transform removes the dominant source of token waste in data-JSON: repeated object keys. An array of N objects that all share the same keys emits each key N times; folding rewrites it to a columnar form that emits each key exactly once.

[ {"id":1,"role":"member"}, {"id":2,"role":"member"} ]
  -> {"__tf_cols__":["id","role"],"__tf_rows__":[[1,"member"],[2,"member"]]}

The fold is applied recursively (values are folded before their enclosing array is), so nested arrays-of-objects fold too. It is a pure structural rewrite — every value is preserved — so unfold_json reconstructs the original data exactly (as a serde_json value; number spelling like 1e10 may be renormalized by re-serialization, but the numeric value is unchanged). The pipeline gates adoption on that round-trip (round_trips), so a fold that would ever lose data is rolled back rather than emitted.

Enums§

JsonFoldError

Constants§

TRANSFORM_ID
Canonical transform id, as registered with the pipeline.
TRANSFORM_VERSION
Semantic version of this transform’s output behavior.

Functions§

fold_json
Folds arrays of homogeneous objects in input into columnar {__tf_cols__, __tf_rows__} form. No-op-safe: empty input returns empty; input with no foldable arrays returns a (compact) re-serialization with the same data.
round_trips
True iff unfolding after reproduces before exactly (as JSON values). The pipeline’s safety gate for this transform — folding is only adopted when this holds.
unfold_json
Inverse of fold_json: expands every {__tf_cols__, __tf_rows__} node back into an array of objects. This is the reversible half that makes the transform safe.