Module fit 

The per-sub-block fitting helpers every K-quant encoder is built from, transcribed from llama.cpp b7650’s ggml/src/ggml-quants.c:

• [nearest_int] (ggml-quants.c:444), the rounding every encoder shares;
• [make_qkx2_quants] (ggml-quants.c:622), the affine scale * L - min fit Q4_K and Q5_K use per 32 weights;
• [make_qx_quants] (ggml-quants.c:451), the symmetric scale * L fit Q6_K uses per 16 weights;
• [fit_qk_super_block], the three-stage Q4_K/Q5_K super-block flow (quantize_row_q4_K_ref at ggml-quants.c:1280, quantize_row_q5_K_ref at ggml-quants.c:1467) that differs between the two formats by exactly four numbers – and, with an importance matrix, the flow of quantize_row_q4_K_impl (:1376) and quantize_row_q5_K_impl (:1581), which is the SAME three stages with a different weight rule, a different candidate grid and a different stage 2 (make_qp_quants, in [super::qp_quants]).

One module, because the alternative is what this repo keeps paying for: a copy of make_qkx2_quants in each of q4_k.rs and q5_k.rs that agree today and drift the first time one of them is corrected. Q5_K is the same super-block fit as Q4_K with nmax = 31 and a different candidate grid, so it is a CALL into the same code, not a second transcription with the constants changed.

The imatrix path is the same rule applied once more. Upstream’s make_qkx3_quants (ggml-quants.c:816) is make_qkx2_quants with a weights ? weights[i] : x[i]*x[i] fallback that no caller in the file exercises (every call passes a weight array) and max <= min for max == min, which cannot differ once min has been clamped to at most zero. So there is ONE [make_qkx2_quants] here, and [fit_qk_super_block] takes the per-super-block importance slice as an Option and switches the three things that actually differ.

Deviation from upstream, shown not to change a byte by the goldens in q4_k, q5_k and q6_k: nearest_int’s assert(fabsf(fval) <= 4194303.f) is not reproduced. It is compiled out of the release libggml that llama-quantize actually links, so asserting here would make ferrox stop where llama.cpp proceeds – a refusal that fires on input llama.cpp handles is not coverage, it is a different tool.

Every mul_add here is load-bearing. Do not “simplify” one.

sumlx += w*x[i]*l in the C is one fused multiply-add, not a multiply followed by an add: the compiler that builds libggml contracts it, so the intermediate product is never rounded to f32. Rust does not contract, so every such site is spelled mul_add explicitly. Writing sumlx += w * x[i] * l as f32 instead is one rounding more, and that rounding is not cosmetic: these fits choose between candidate scales with sumlx*sumlx > best*suml2, a comparison that is a near-tie often enough that ONE ulp flips which candidate wins and rewrites the whole super-block.

Measured on an F16 Llama-3.2-1B, against the installed llama-quantize b7650: with these mul_adds, ferrox writes byte-identical files – 0 of 3244032 Q4_K super-blocks differ, 0 of 3244032 Q5_K, 0 of 4827136 Q6_K. Remove them and it is 1.15%, 0.15% and 1.39% respectively. That is the entire difference between “a file llama.cpp would have written” and “a file that decodes to similar numbers”.

Nine of the thirteen sites have a real-weight super-block in testdata::REAL_WEIGHT_BLOCKS that turns a golden red when that one mul_add is removed; the fixture doc names the four that do not and why. Synthetic noise pins NONE of them, which is how they were nearly shipped wrong.