Module kv_cache 

KV cache — per-layer, head-major storage.

Layout: one contiguous Vec<f32> per KV head ([pos × head_dim]), so per-head attention reads a straight slice — no per-head gather copies per token. Dead GQA groups (all Q heads masked) store nothing at all: masked heads cost neither FLOPs nor memory.

Structs

KvCache

Full KV cache for all layers.

LayerKvCache

KV cache for a single layer, head-major.

Enums

EvictionPolicy

Eviction policy for a bounded cache.

KvMode

KV storage mode. CMF_KV=q8 enables the q8_2f cache: an int8 row per (position, head) + an f32 scale per row + a per-channel field 𝒲×θ, frozen after WARMUP positions with retroactive requantization (D4: “KV-quant 2f”). Memory ×~3.7 smaller than f32.