Struct ContiguousKVCache 

pub struct ContiguousKVCache<B: Backend> { /* private fields */ }

Simple contiguous cache: stores one K and one V tensor per layer and concatenates along the sequence dimension every step.

Cost: an O(seq) copy per token per layer — kept as the correctness baseline; the paged arena is the production default.

Implementations

impl<B: Backend> ContiguousKVCache<B>

pub fn new(num_layers: usize) -> Self

Creates an empty cache for num_layers layers.

Trait Implementations

impl<B: Backend> KVCache<B> for ContiguousKVCache<B>

fn attention( &mut self, layer: usize, q: Tensor<B, 4>, k: Tensor<B, 4>, v: Tensor<B, 4>, pos: usize, scale: f64, ) -> Tensor<B, 4>

Appends seq new positions of K/V for layer and computes attention of q against the full cached window (past + new).

fn seq_len(&self) -> usize

Total cached sequence length.

fn reset(&mut self)

Drops all cached state (session reset).

fn popn(&mut self, n: usize) -> usize

Rolls back the last n cached tokens, returning how many were actually dropped. Caches that cannot roll back (the contiguous baseline) return 0 — callers gate prefix reuse on a nonzero result.

fn pages_used(&self) -> Option<usize>

Pages currently allocated to the sequence (paged cache only).