Module encode 

Weight encoders: f32 in, GGUF block bytes out.

The rest of this crate reads quantized blocks. This module is the only place that writes them, and today it writes four formats: Q8_0 here, and Q4_K, Q5_K and Q6_K in q4_k, q5_k and q6_k. The rest is not an oversight, it is the scope: llama.cpp’s remaining K-quant and IQ encoders each need their own transcription (and, for the IQ tiers, a lattice search over a codebook), and a naive min/max encoder wearing a K-quant’s name produces a file that loads and generates measurably worse text. ferrox quantize refuses every target this module cannot encode, by name.

The three K-quants share ONE transcription of the per-sub-block fit, in fit. Q4_K and Q5_K differ by four numbers in a QkFit, not by a second copy of make_qkx2_quants; Q6_K reaches the same module for nearest_int and make_qx_quants. Two copies of a fit that must agree is this repo’s dominant bug shape, and a K-quant encoder is about the worst place to have one: the copies would agree the day they were written and diverge invisibly, since both would still dequantize to plausible weights.

Each format lands with a byte-identical golden against llama.cpp’s own encoder, never a tolerance: two encoders can agree on dequantized values and still write different files.

Q8_0 here is byte-for-byte llama.cpp’s quantize_row_q8_0_ref, not merely “close enough”. The arithmetic below is deliberately the same shape as the C, including the reciprocal multiply and the a > b ? a : b maximum, because the file this writes is meant to be indistinguishable from llama-quantize --type Q8_0’s. See q8_0_matches_llama_cpp_quantize_row_q8_0_ref for the golden.

Modules

fit

The per-sub-block fitting helpers every K-quant encoder is built from, transcribed from llama.cpp b7650’s ggml/src/ggml-quants.c:

q4_k

The Q4_K weight encoder: a transcription of llama.cpp b7650’s quantize_row_q4_K_ref (ggml/src/ggml-quants.c:1280), not a reimplementation of it.

q5_k

The Q5_K weight encoder: a transcription of llama.cpp b7650’s quantize_row_q5_K_ref (ggml/src/ggml-quants.c:1467).

q6_k

The Q6_K weight encoder: a transcription of llama.cpp b7650’s quantize_row_q6_K_ref (ggml/src/ggml-quants.c:1692).

Functions

encode_block_q8_0

Encodes one Q8_0 block (exactly Q8_0_BLOCK_ELEMS values) and appends its Q8_0_BLOCK_BYTES bytes to out.

encode_row_q8_0

Encodes a whole row (or any slice whose length is a multiple of Q8_0_BLOCK_ELEMS) into Q8_0 blocks, appending to out.