Expand description
Weight encoders: f32 in, GGUF block bytes out.
The rest of this crate reads quantized blocks. This module is the
only place that writes them, and today it writes exactly one format.
That is not an oversight, it is the scope: llama.cpp’s K-quant and
IQ encoders do an iterative scale/min fit (and, for the IQ tiers, a
lattice search), and a naive min/max encoder wearing a K-quant’s
name produces a file that loads and generates measurably worse text.
ferrox quantize refuses every target this module cannot encode, by
name.
Q8_0 here is byte-for-byte llama.cpp’s quantize_row_q8_0_ref,
not merely “close enough”. The arithmetic below is deliberately the
same shape as the C, including the reciprocal multiply and the
a > b ? a : b maximum, because the file this writes is meant to be
indistinguishable from llama-quantize --type Q8_0’s. See
q8_0_matches_llama_cpp_quantize_row_q8_0_ref for the golden.
Functions§
- encode_
block_ q8_ 0 - Encodes one Q8_0 block (exactly
Q8_0_BLOCK_ELEMSvalues) and appends itsQ8_0_BLOCK_BYTESbytes toout. - encode_
row_ q8_ 0 - Encodes a whole row (or any slice whose length is a multiple of
Q8_0_BLOCK_ELEMS) into Q8_0 blocks, appending toout.