pub enum KvMode {
F32,
Q8 {
k: bool,
v: bool,
},
}Expand description
KV storage mode. CMF_KV=q8 enables the q8_2f cache: an int8 row per
(position, head) + an f32 scale per row + a per-channel scale field,
frozen after WARMUP positions with retroactive requantization
(D4: “KV-quant 2f”). Memory ×~3.7 smaller than f32.
Variants§
Implementations§
Trait Implementations§
impl Copy for KvMode
impl Eq for KvMode
impl StructuralPartialEq for KvMode
Auto Trait Implementations§
impl Freeze for KvMode
impl RefUnwindSafe for KvMode
impl Send for KvMode
impl Sync for KvMode
impl Unpin for KvMode
impl UnsafeUnpin for KvMode
impl UnwindSafe for KvMode
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more