Module q4_k 

The Q4_K weight encoder: a transcription of llama.cpp’s quantize_row_q4_K_ref (ggml/src/ggml-quants.c), not a reimplementation of it.

A K-quant is NOT min/max over a block. Q4_K’s 256-element super-block is fitted in three stages, and every one of them has to be reproduced exactly or the file differs:

1. Each of the 8 sub-blocks of 32 gets an iterative affine fit (make_qkx2_quants): 21 candidate inverse scales are tried, each one re-solves a weighted least-squares for (scale, min) from the integer codes it produced, and the lowest weighted squared error wins. The weights are sqrt(mean(x^2)) + |x|, so a sub-block’s large values pull the fit toward themselves.
2. The 8 scales and 8 mins are themselves quantized to 6 bits against the super-block’s d/dmin and packed into 12 bytes.
3. The 4-bit codes are then recomputed against the 6-bit-rounded scale and min, not against the fit from stage 1 – so stage 3 sees a slightly different affine map than stage 1 did.

A naive min/max encoder skips all three and produces a file that loads and generates measurably worse text. That is the failure this module exists to not ship, so the arithmetic below is deliberately the same shape as the C, down to the operation order in the least-squares accumulation.

Deviations from upstream, all of them shown not to change a byte by q4_k_matches_llama_cpp_quantize_row_q4_k_ref in tests:

• nearest_int’s assert(fabsf(fval) <= 4194303.f) is not reproduced. It is compiled out of the release libggml that llama-quantize actually links, so asserting here would make ferrox stop where llama.cpp proceeds – a refusal that fires on input llama.cpp handles is not coverage, it is a different tool.
• The 6-bit scale/min are unpacked for stage 3 by the same [crate::q4_k_scale_min] the reader uses, rather than by a second copy of get_scale_min_k4. Two copies of that bit-packing is precisely the shape of bug this repo keeps finding; one function means the encoder and the decoder cannot disagree about what was packed.

Functions

encode_block_q4_k

Encodes one Q4_K super-block (exactly Q4_K_BLOCK_ELEMS values) and appends its Q4_K_BLOCK_BYTES bytes to out.

encode_row_q4_k

Encodes a whole row (or any slice whose length is a multiple of Q4_K_BLOCK_ELEMS) into Q4_K super-blocks, appending to out.