Struct Scratchpad 

pub struct Scratchpad { /* private fields */ }

A pre-allocated scratch buffer for KV-cache. Allows appending new key/value vectors without re-allocation, up to a fixed max_seq_len.

Layout: [max_seq_len, dim] with a current_len cursor. All memory is allocated once at construction; append only copies new data into existing storage (zero GC pressure per token).

Implementations

impl Scratchpad

pub fn new(max_seq_len: usize, dim: usize) -> Self

Create a new scratchpad pre-allocated for max_seq_len tokens of dimension dim. Zero-fills all storage upfront.

pub fn len(&self) -> usize

Number of tokens currently stored.

pub fn is_empty(&self) -> bool

Whether no tokens are stored.

pub fn capacity(&self) -> usize

Maximum sequence length this scratchpad can hold.

pub fn dim(&self) -> usize

Hidden dimension per token.

pub fn append(&mut self, token_vec: &[f64]) -> Result<(), RuntimeError>

Append a single token vector [dim] to the cache. Returns an error if the cache is full. Zero allocation.

pub fn append_tensor(&mut self, t: &Tensor) -> Result<(), RuntimeError>

Append a batch of token vectors from a tensor of shape [n, dim]. Zero allocation — writes directly into pre-allocated storage.

pub fn as_tensor(&self) -> Tensor

Get a Tensor view [current_len, dim] of the stored data. Shares the underlying buffer (zero-copy).

pub fn clear(&mut self)

Reset the cache to empty without deallocating. The underlying buffer is retained for reuse.

Trait Implementations

impl Clone for Scratchpad

fn clone(&self) -> Scratchpad

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Scratchpad

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Display for Scratchpad

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.