Struct KvCacheSnapshot 

pub struct KvCacheSnapshot {
    pub keys: Vec<Vec<f32>>,
    pub values: Vec<Vec<f32>>,
    pub seq_len: usize,
}

A point-in-time snapshot of a KvCache state.

Created by KvCache::snapshot and restored via KvCache::restore_from_snapshot. Used by crate::speculative::SpeculativeDeltaSync to roll back the KV cache after a draft token is rejected.

Fields

keys: Vec<Vec<f32>>

Per-layer key vectors, each of length seq_len * kv_dim.

values: Vec<Vec<f32>>

Per-layer value vectors, each of length seq_len * kv_dim.

seq_len: usize

The sequence length at snapshot time.

Trait Implementations

impl Clone for KvCacheSnapshot

fn clone(&self) -> KvCacheSnapshot

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for KvCacheSnapshot

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Decode for KvCacheSnapshot

fn decode<__D: Decoder<Context = ()>>(decoder: &mut __D) -> Result<Self, Error>

Decode a value from the given decoder

impl Encode for KvCacheSnapshot

fn encode<__E: Encoder>(&self, encoder: &mut __E) -> Result<(), Error>

Encode this value into the given encoder