Skip to main content

Module encode

Module encode 

Source
Expand description

Weight encoders: f32 in, GGUF block bytes out.

The rest of this crate reads quantized blocks. This module is the only place that writes them, and today it writes exactly one format. That is not an oversight, it is the scope: llama.cpp’s K-quant and IQ encoders do an iterative scale/min fit (and, for the IQ tiers, a lattice search), and a naive min/max encoder wearing a K-quant’s name produces a file that loads and generates measurably worse text. ferrox quantize refuses every target this module cannot encode, by name.

Q8_0 here is byte-for-byte llama.cpp’s quantize_row_q8_0_ref, not merely “close enough”. The arithmetic below is deliberately the same shape as the C, including the reciprocal multiply and the a > b ? a : b maximum, because the file this writes is meant to be indistinguishable from llama-quantize --type Q8_0’s. See q8_0_matches_llama_cpp_quantize_row_q8_0_ref for the golden.

Functions§

encode_block_q8_0
Encodes one Q8_0 block (exactly Q8_0_BLOCK_ELEMS values) and appends its Q8_0_BLOCK_BYTES bytes to out.
encode_row_q8_0
Encodes a whole row (or any slice whose length is a multiple of Q8_0_BLOCK_ELEMS) into Q8_0 blocks, appending to out.