Struct WindowKvCache 

pub struct WindowKvCache {
    pub axis: usize,
    pub window: usize,
    /* private fields */
}

Fixed-capacity sliding-window KV cache (ring buffer) along axis.

Fields

axis: usize

window: usize

Implementations

impl WindowKvCache

pub fn new(axis: usize, window: usize) -> Self

pub fn len(&self) -> usize

pub fn is_empty(&self) -> bool

pub fn capacity(&self) -> usize

Always the window capacity once allocated — memory is bounded regardless of T.

pub fn push(&mut self, input: &Tensor) -> TractResult<()>

Append input along axis, overwriting the oldest slots once full. O(min(new, window)).

pub fn valid_view<T: Datum>(&self) -> TractResult<ArrayViewD<'_, T>>

Zero-copy view of the len valid slots. Once full this is the whole buffer in physical (rotated) order — correct for decode attention by order-invariance.

Trait Implementations

impl Clone for WindowKvCache

fn clone(&self) -> WindowKvCache

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for WindowKvCache

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.