Trait QuantizedKVCache 

pub trait QuantizedKVCache {
    // Required methods
    fn store_key(&mut self, layer: usize, pos: usize, key: &[f32]);
    fn store_value(&mut self, layer: usize, pos: usize, value: &[f32]);
    fn dequantize_key_into(&mut self, layer: usize, pos: usize, out: &mut [f32]);
    fn dequantize_value_into(
        &mut self,
        layer: usize,
        pos: usize,
        out: &mut [f32],
    );
    fn reset(&mut self);
    fn pos(&self) -> usize;
    fn set_pos(&mut self, pos: usize);
}

Shared interface for quantized KV caches.

Enables transformer::forward_quantized to work with any compression backend (TurboQuant, SpectralQuant, OscKV, ShardKV, KVarN, or future methods). Backends implement this trait; the inference loop stays backend-agnostic.

Required Methods

fn store_key(&mut self, layer: usize, pos: usize, key: &[f32])

Quantize and store a key vector at given layer and position.

fn store_value(&mut self, layer: usize, pos: usize, value: &[f32])

Quantize and store a value vector at given layer and position.

fn dequantize_key_into(&mut self, layer: usize, pos: usize, out: &mut [f32])

Dequantize a key into a pre-allocated buffer (zero-alloc hot path).

fn dequantize_value_into(&mut self, layer: usize, pos: usize, out: &mut [f32])

Dequantize a value into a pre-allocated buffer (zero-alloc hot path).

fn reset(&mut self)

Reset cache for a new sequence.

fn pos(&self) -> usize

Current write position.

fn set_pos(&mut self, pos: usize)

Set the current write position.

Dyn Compatibility

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors