Module kv_cache 

KV cache — per-layer, head-major storage.

Layout: one contiguous Vec<f32> per KV head ([pos × head_dim]), so per-head attention reads a straight slice — no per-head gather copies per token. Dead GQA groups (all Q heads masked) store nothing at all: masked heads cost neither FLOPs nor memory.

Structs

KvCache

Full KV cache for all layers.

LayerKvCache

KV cache for a single layer, head-major.

Enums

EvictionPolicy

Eviction policy for a bounded cache.

KvMode

KV storage mode. CMF_KV=q8 enables the q8_2f cache: an int8 row per (position, head) + an f32 scale per row + a per-channel scale field, frozen after WARMUP positions with retroactive requantization (D4: “KV-quant 2f”). Memory ×~3.7 smaller than f32.

O1State

Per-layer O(1) Nyström attention state (runtime attn_type override — spec §7 presence-driven pattern, no format change).

WireKind

State-record kind of the versioned cache wire (export_wire v2).

Constants

WIRE_MAGIC

Magic of the versioned state wire.

WIRE_VERSION

Current wire version.