Expand description
json_value_dict transform (canonical id "json_value_dict", v1.0.0).
Content-aware, losslessly reversible value deduplication for generic JSON data.
Where json_field_fold factors out repeated keys, this factors out repeated values:
the same large object/array/string appearing many times (a constant nested record, a
repeated timestamp, an identical config blob) is stored once in a dictionary and every
occurrence is replaced by a compact reference.
{"a":{"x":1,"y":2,"z":3},"b":{"x":1,"y":2,"z":3}}
-> {"__tf_dict__":[{"x":1,"y":2,"z":3}],
"__tf_data__":{"a":{"__tf_ref__":0},"b":{"__tf_ref__":0}}}It runs after json_field_fold in the pipeline, so it also collapses the repeated
nested values that folding surfaces across rows. Only values whose serialized form is
large enough that a reference is cheaper are dictionaried (see MIN_VALUE_BYTES), and the
pipeline’s exact-token gate drops the whole transform if it fails to net a saving — so it
can never make a payload worse. Reversibility is guaranteed by round_trips (the pipeline
safety gate): a fold that wouldn’t restore exactly is rolled back.
Enums§
Constants§
- TRANSFORM_
ID - Canonical transform id, as registered with the pipeline.
- TRANSFORM_
VERSION - Semantic version of this transform’s output behavior.
Functions§
- dict_
json - Replaces repeated large values in
inputwith dictionary references. No-op-safe: empty input returns empty; input with nothing worth dictionarying returns unchanged. - round_
trips - True iff expanding
afterreproducesbeforeexactly (as JSON values) — the pipeline’s safety gate for this transform. - undict_
json - Inverse of
dict_json: expands every{__tf_ref__: i}back todict[i].