Struct KvCacheState 

pub struct KvCacheState { /* private fields */ }

Tracks the token sequence currently held in the KV cache.

By comparing a new prompt’s token sequence against the cached one, the agent loop can skip re-encoding the common prefix and only decode the delta. This is especially beneficial for multi-agent conversations where ~95% of the prompt is unchanged between turns.

Implementations

impl KvCacheState

pub fn new() -> Self

pub fn invalidate(&mut self)

Invalidate the cache (e.g. after compaction or history clear).

pub fn len(&self) -> usize

pub fn is_empty(&self) -> bool

Trait Implementations

impl Default for KvCacheState

fn default() -> Self

Returns the “default value” for a type.