Expand description
IQ2_XXS, IQ2_XS, IQ2_S, IQ3_XXS, IQ3_S, IQ1_S and IQ1_M block
formats (GAP-211 B2): the grid-table IQ quantizations. DECODE-ONLY.
§Provenance
Block layouts and dequantization are llama.cpp’s / ggml’s (MIT License,
Copyright (c) 2023-2026 The ggml authors), checked against the source at
commit 9d57ce456c (the pin recorded on GgmlDType):
ggml/src/ggml-common.h:block_iq2_xxs,block_iq2_xs,block_iq2_s,block_iq3_xxs,block_iq3_s,block_iq1_s,block_iq1_m,IQ1S_DELTA.ggml/src/ggml-quants.c:dequantize_row_iq2_xxs…dequantize_row_iq1_m.- the lookup tables themselves are in
crate::iq_grids, generated fromggml-common.hbyscripts/gen_iq_grids.py.
§Decode-only
llama.cpp’s quantizers for these formats search for grid points under an
importance matrix and are out of scope here. GgmlType::from_float is
therefore not provided: it panics naming the type, and the fallible
quantize entry points refuse these dtypes first
(GgmlDType::supports_quantize). A model stored in these formats loads,
dequantizes and multiplies (dequantize-then-dot); it cannot be produced here.