Struct KVCacheConfig 

pub struct KVCacheConfig {
    pub max_seq_len: usize,
    pub num_heads: usize,
    pub head_dim: usize,
    pub quantization_bits: u8,
    pub eviction_policy: EvictionPolicy,
}

Configuration for the quantized KV-cache.

Fields

max_seq_len: usize

Maximum sequence length the cache can hold before eviction is required.

num_heads: usize

Number of attention heads.

head_dim: usize

Dimension per attention head.

quantization_bits: u8

Bit-width for quantization. Supported: 2, 3, 4, 8.

eviction_policy: EvictionPolicy

Policy used when the cache exceeds its budget.

Trait Implementations

impl Clone for KVCacheConfig

fn clone(&self) -> KVCacheConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for KVCacheConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.