Struct KvCacheMps 

pub struct KvCacheMps { /* private fields */ }

MPS-compressed KV cache backed by independent online low-rank bases for keys and values.

Implementations

impl KvCacheMps

pub fn new(d_k: usize, d_v: usize, chi_max: usize) -> Self

Create an empty cache.

d_k / d_v: key / value head dimensions. chi_max: maximum bond dimension (compression rank).

pub fn append(&mut self, k: &[f32], v: &[f32])

Append a new (k, v) token pair.

If the respective basis is not yet full (chi < chi_max), a new orthonormal direction is added via Gram-Schmidt; otherwise the token is projected onto the existing basis (lossy compression).

pub fn token_count(&self) -> usize

Number of tokens stored.

pub fn max_bond_dim(&self) -> usize

Current bond dimension (max of key and value ranks; ≤ chi_max).

pub fn attend(&self, query: &[f32], scale: f32) -> Vec<f32>

Compute softmax attention over the compressed KV cache.

query has length d_k. Returns a vector of length d_v.

Scores are computed as (B_k^T · query) · c_k[t], which equals the dot product query · K_approx[t] exactly (the basis is orthonormal).

pub fn compression_ratio(&self) -> f64

Compression ratio: flat storage bytes / compressed storage bytes.

Values > 1 mean the cache uses less memory than storing KV pairs flat. The ratio grows with T; for large T it approaches d_kv / chi_max.