Skip to main content

Module schema_codec

Module schema_codec 

Source
Expand description

Schema-aware structural compression for fixed-width bridge slots.

The bridge ships fixed-width repr(C) slots, and measurement on the real slot types shows 40-75% of every structured slot is bytes that never carry information: padding, reserved fields, stable high bytes of small enums, and counts. Those byte positions are constant across the stream. This codec learns which positions are constant (a template negotiated once, the “mask in one packet”), then ships only the bytes at the varying positions. The receiver scatters them back into the template.

It is exact, not lossy: a slot whose supposedly-constant position actually differs (the escape) is shipped in full under an escape flag, so round-trip is byte-identical for any input, and the constant model is a throughput optimization that can never corrupt.

It is cache-resident: encode is a linear walk of a precomputed constant-position list (the escape check) plus a linear gather of a precomputed varying-position list; both lists are small u16 vectors that stay in L1. There is no per-slot allocation. Stream-level parallelism rides the shard threads, one template per shard.

Structs§

SchemaTemplate
A learned constant-position template for a fixed-width slot stream.