Enum KvElem 

pub enum KvElem {
    F32,
    F16,
    Q8_0,
    Turbo4,
}

Element width of one cached K/V scalar, per backend store.

The block-quantized variants are the ggml/TurboQuant wire formats ferrox-metal writes for FERROX_CTK (see ferrox_metal::attn::MetalKvDtype), so their cost is per 32-element block, not per scalar.

Variants

F32

Host ferrox_core::cache::KvCache, which stores Vec<f32>.

F16

Metal device KV default (FERROX_CTK=f16, llama.cpp -ctk f16).

Q8_0

ggml Q8_0 wire: 32 elems -> 2-byte scale + 32 int8 = 34 bytes. FERROX_CTK=q8_0|turbo8|fp8 all land on this width.

Turbo4

TurboQuant 4-bit: 32 elems -> 2-byte scale + 16 nibble bytes.

Implementations

impl KvElem

pub fn bytes_for(self, elems: u64) -> u64

Bytes needed to store elems cached scalars, rounding up to a whole block for the block-quantized wires (a partial block still costs a full one).

pub fn as_str(self) -> &'static str

pub fn from_ctk(value: &str) -> Self

Maps a FERROX_CTK / --ctk value onto the width the Metal KV store really keeps. Mirrors ferrox_metal::attn::effective_metal_kv_dtype: turbo8 and fp8 share Q8_0’s 34-byte wire, and anything unrecognised or unimplemented (turbo3) falls back to f16 rather than being budgeted at a width no kernel writes.

Note this does not check the block alignment that function also checks (n_kv_heads * head_dim divisible by 32), so a misaligned shape is budgeted at the requested width while the runtime silently uses f16 – an under-estimate, called out here rather than papered over.

Trait Implementations

impl Clone for KvElem

fn clone(&self) -> KvElem

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for KvElem

impl Debug for KvElem

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for KvElem

impl PartialEq for KvElem

fn eq(&self, other: &KvElem) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for KvElem