Struct QuantizedKvLayer 

pub struct QuantizedKvLayer {
    pub k: Vec<u8>,
    pub v: Vec<u8>,
    pub past_len: usize,
    pub kv_dim: usize,
    pub scheme: KvQuant,
}

One layer’s quantized K/V buffers.

Rows are appended back-to-back. past_len is the number of logical rows currently stored; the byte buffers carry past_len × kv_dim elements’ worth of quantized bytes.

Fields

k: Vec<u8>

v: Vec<u8>

past_len: usize

kv_dim: usize

scheme: KvQuant

Implementations

impl QuantizedKvLayer

pub fn new(kv_dim: usize, scheme: KvQuant) -> Result<Self>

pub fn append_rows(&mut self, k_rows: &[f32], v_rows: &[f32]) -> Result<()>

Append rows worth of K and V (f32, row-major) — quantizing each row independently. Caller passes interleaved K then V blocks via separate slices.

pub fn read_all(&self) -> Result<(Vec<f32>, Vec<f32>)>

Dequantize all stored rows back to f32 (K, V).

pub fn read_window(&self, window: usize) -> Result<(Vec<f32>, Vec<f32>)>

Dequantize the last window rows (or all rows if past_len ≤ window).

pub fn drop_front(&mut self, n_rows: usize) -> Result<()>

Drop the oldest n_rows from this layer (sliding window).

pub fn bytes(&self) -> usize

Memory used by both buffers (bytes).

Trait Implementations

impl Clone for QuantizedKvLayer

fn clone(&self) -> QuantizedKvLayer

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for QuantizedKvLayer

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.