Enum KvCacheDtype 

pub enum KvCacheDtype {
    Fp16,
    Bf16,
    Int8,
    Fp8,
}

KV Cache element dtype (Dim 5 polymorphism point).

Mirrors ferrum_interfaces::kv_dtype::KvDtypeKind markers but lives here because KvCacheConfig is part of the user-facing EngineConfig and needs Serialize / Deserialize.

Variants

Fp16

FP16 K/V — the validated production path on every backend.

Bf16

BF16 K/V — same memory cost as FP16, slightly different precision. Marker only; no backend impl ships yet.

Int8

INT8 K/V with per-token per-kv-head FP16 scale (vLLM-style). Halves KV memory at small (<1%) accuracy hit. CUDA kernels land via BackendKvDtype<KvInt8> (PR #131); model wire-up (KvCacheQuant<B, KvInt8> through the model decode loop) is the only remaining step.

Fp8

FP8 (E4M3) K/V. Marker only; CUDA kernels pending.

Implementations

impl KvCacheDtype

pub fn parse(s: &str) -> Option<Self>

Parse from a CLI / env-var string. Accepts fp16/f16/bf16/int8/fp8/f8e4m3.

pub fn as_str(&self) -> &'static str

Short label for display + telemetry.

Trait Implementations

impl Clone for KvCacheDtype

fn clone(&self) -> KvCacheDtype

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for KvCacheDtype

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for KvCacheDtype

fn default() -> KvCacheDtype

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for KvCacheDtype

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl PartialEq for KvCacheDtype

fn eq(&self, other: &KvCacheDtype) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Serialize for KvCacheDtype

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl Copy for KvCacheDtype

impl Eq for KvCacheDtype

impl StructuralPartialEq for KvCacheDtype