Module kvcodec 

KV-cache codec: append-friendly, error-bounded coding of attention K/V.

Input is one layer’s cache, [heads][tokens][head_dim] f32. The design follows the statistics of real caches (SmolLM2 dumps, see bench/kv_ppl.py) rather than treating the cache as a generic 3-D array:

• K is coded pre-RoPE. The rotation is undone with the stream’s theta and absolute token position, coded, and re-applied on decode. Pre-RoPE, most of K’s energy is a per-channel constant; post-RoPE that constant becomes a position-dependent oscillation. RoPE is a per-pair rotation, so a pre-RoPE pair error of e is a post-RoPE pair error of e too, and every per-channel quantity is shared by the two channels of a RoPE pair (c, c + hd/2).
• Prediction runs along the token axis only, per (head, channel): closed-loop DPCM around the channel mean with a per-channel AR(1) coefficient, uniform step. Closed loop means every reconstructed value is within step/2 of the (pre-RoPE) input — the error bound is exact.
• Attention sinks (the first sink absolute positions) are coded with a sink_div-times finer step: they carry most of the attention mass, so their error is amplified far beyond their share of the values.
• Context models are per head: each residual is coded with adaptive binary models selected by (head, channel spread class, magnitude of the same channel’s previous residual). Models start from a discrete-Laplace prior of the class’s spread, so a fresh block learns nothing from zero.
• Blocks of block tokens are independent (prediction and models reset), so new tokens are coded without touching the past and any token range decodes by decoding only the blocks that cover it. The stream statistics (channel means, AR coefficients, classes) are fixed from the first chunk the encoder sees (the prefill).
• GPU layout (gpu, header flag bit 3): reconstruction is integer (values are i32 multiples of the unit u = step / sink_div, stored as f32; the AR prediction is mu + (rho·(prev − mu)) >> 4), and every block is coded as one independent arithmetic-coded substream per head. The (block, head) substreams are the GPU decoder’s lanes (crate::kvgpu), and its output is bit-identical to KvStream::decode_range: the only float operation is the final (q as f32) * u. The layout codes the coded domain directly (the caller passes pre-RoPE K and applies RoPE itself).

Structs

GpuTables

Per-stream tables the GPU decoder uploads (see KvStream::gpu_tables).

KvEncoder

Append-friendly encoder for one layer’s K or V stream.

KvParams

Encoder settings.

KvStream

Parsed stream: header plus block payload slices.

Enums

Kind

Which half of the cache a stream holds.

Constants

GPU_CTXS

Contexts per lane (classes × previous-magnitude bins).

GPU_CTX_MODELS

Models per context in gpu_prior_table.

Functions

decode

Decode a whole stream → (params, tokens, [heads][tokens][hd]).

encode

One-shot encode of [heads][n][hd] (prefill = everything).

gpu_prior_table

Initial adaptive-model table shared by every GPU-layout stream: [class][act][49] with entry 0 the zero flag, 1..=24 the unary models and 25..=48 the mantissa models, each p | p << 16 (fast, slow).

lane_chan

Channel of lane-local index i in channel group g (see KvParams::lane_ch).

step_for_bpv

Rate-targeted mode: the step whose one-shot stream is closest to (and not above) bpv bits per value, by bisection on log(step).