Skip to main content

FP8_KV_GROUP

ferrox_quant

Constant FP8_KV_GROUP 

Source 
pub const FP8_KV_GROUP: usize = 32;
Expand description

Metal FERROX_CTK=fp8 KV block: 32 elems → f16 scale + 32 E4M3-ish bytes. Codes are absmax-scaled int8 in [-127,127] (portable stand-in for E4M3).