Skip to main content

Module kv_cache

Module kv_cache 

Source
Expand description

Per-layer K/V cache for autoregressive decode (Whisper, Qwen, Gemma, …).

Structs§

LayerKvCache
Layer-wise past K/V tensors in row-major [past_len * kv_dim] layout per layer.