Struct LayerKvCache 

pub struct LayerKvCache {
    pub mode: KvMode,
    pub seq_len: usize,
    pub num_kv_heads: usize,
    pub head_dim: usize,
    pub linear_state: Vec<f64>,
    pub linear_scratch: Vec<f64>,
    /* private fields */
}

KV cache for a single layer, head-major.

Fields

mode: KvMode

seq_len: usize

Positions appended so far (grows once per token, dead heads included).

num_kv_heads: usize

head_dim: usize

linear_state: Vec<f64>

Linear-core condensate S (vmf_phase), f64; empty on full layers.

linear_scratch: Vec<f64>

Tentative lane-2 state during speculative verify.

Implementations

impl LayerKvCache

pub fn new(num_kv_heads: usize, head_dim: usize) -> Self

pub fn append(&mut self, k_new: &[f32], v_new: &[f32], alive: &[bool])

Append K/V for one position. k_new/v_new are [num_kv_heads × head_dim]; heads with alive[h] == false are skipped (their slices stay empty).

pub fn attend(&self, q: &[f32], kv_head: usize) -> (Vec<f32>, Vec<f32>)

Per-head attention over its own storage: the f32 branch is bit-for-bit equal to attention_head() over slices; the q8 branch computes score = s_k·⟨q⊙col_k, k_q⟩ and the weighted sum of V in i8 with f32 accumulation. Returns (output [head_dim], probs [stored]).

pub fn truncate_last(&mut self, n_drop: usize)

Roll back the last n_drop positions (speculative-decode reject).

pub fn accumulate_imp(&mut self, probs: &[f32])

Accumulate attention mass per stored position (summed over heads).

pub fn head_keys(&self, kv_head: usize) -> &[f32]

Contiguous keys of one head: [stored_len × head_dim].

pub fn head_values(&self, kv_head: usize) -> &[f32]

pub fn head_len(&self, kv_head: usize) -> usize

Number of positions actually stored for a head (0 for dead heads).

pub fn clear(&mut self)

Clear cache (e.g. on new conversation or task switch).

pub fn memory_bytes(&self) -> usize

Memory usage in bytes.

Trait Implementations

impl Clone for LayerKvCache

fn clone(&self) -> LayerKvCache

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for LayerKvCache

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.