Struct InPlaceKvCache 

pub struct InPlaceKvCache {
    pub axis: usize,
    /* private fields */
}

Geometric-growth in-place cache along axis.

Fields

axis: usize

Implementations

impl InPlaceKvCache

pub fn new(axis: usize) -> Self

pub fn len(&self) -> usize

pub fn is_empty(&self) -> bool

pub fn reallocs(&self) -> usize

Number of reallocations so far (should be O(log T) over a decode).

pub fn capacity(&self) -> usize

pub fn push(&mut self, input: &Tensor) -> TractResult<()>

Append input along axis, in place. O(input) amortized.

pub fn valid_view<T: Datum>(&self) -> TractResult<ArrayViewD<'_, T>>

Zero-copy ndarray view of the valid [0..len] region. This is the path that realizes the win — a length-aware consumer attends over this without ever copying the past.

pub fn valid_contiguous(&self) -> TractResult<Tensor>

Contiguous [0..len] copy. This is what a consumer needing an owned, packed tensor forces — it re-introduces the per-step O(len) copy, so it is only kept to A/B against valid_view.

Trait Implementations

impl Clone for InPlaceKvCache

fn clone(&self) -> InPlaceKvCache

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for InPlaceKvCache

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.