Enum KvPrecision 

pub enum KvPrecision {
    F32,
    Int8,
}

How the key/value cache stores its entries.

The container declares what the model was post-trained for in kv_bits. Needle 3 declares 8, so KvPrecision::Int8 is the width upstream intends; < 8 maps to a different scheme upstream (Cactus-Quants at group 64) and is not implemented here, because no shipped checkpoint asks for it.

Variants

F32

Store as f32. Bit-identical to the verified forward pass, and the default for that reason.

Int8

Per-head symmetric 8-bit, as upstream’s a8_fake_quant_kv defines it.

Roughly a quarter of the memory. It is not bit-identical to the f32 path — it is the numerics upstream’s native engine runs — so it is opt-in rather than default.

Trait Implementations

impl Clone for KvPrecision

fn clone(&self) -> KvPrecision

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for KvPrecision

impl Debug for KvPrecision

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for KvPrecision

impl PartialEq for KvPrecision

fn eq(&self, other: &KvPrecision) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for KvPrecision