Enum KvQuant 

pub enum KvQuant {
    F16,
    Q8_0,
    Q4_0,
    Q5_0,
}

Quantization scheme for cache rows. Restricted to the three q-formats whose blocks are 32 elements wide and stable across llama.cpp versions. The K-quants (Q4_K etc.) require 256-element blocks, which doesn’t compose cleanly with typical kv_dim values (e.g. 128 head dim) so we don’t expose them here.

Variants

F16

f16 — lossless storage of f32→f16 (no quantization). Kept as a useful baseline; 2 bytes per element.

Q8_0

Q4_0

Q5_0

Implementations

impl KvQuant

pub const fn block_elements(self) -> usize

On-disk block size in elements.

pub const fn block_bytes(self) -> usize

On-disk block size in bytes.

pub fn bytes_for(self, n_elements: usize) -> Result<usize>

Bytes required to store n_elements quantized.

Trait Implementations

impl Clone for KvQuant

fn clone(&self) -> KvQuant

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for KvQuant

impl Debug for KvQuant

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for KvQuant

impl PartialEq for KvQuant

fn eq(&self, other: &KvQuant) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for KvQuant