Enum QuantKind 

pub enum QuantKind {
    Q8_0,
    Q4_0,
    Q4K,
    Q5K,
    Q6K,
    Q2K,
    Q3K,
    Q4_1,
    Q5_0,
    Q5_1,
    Q8_1,
    IQ4NL,
    IQ4XS,
    IQ1S,
    IQ2XXS,
    IQ3XXS,
    IQ2XS,
    IQ2S,
    IQ3S,
    IQ1M,
    Mxfp4Gguf,
}

Variants

Q8_0

Q4_0

Q4K

The dominant real-world GGUF quantization formats (most published checkpoints ship as Q4_K_M or similar K-quant mixes, not the legacy Q4_0/Q8_0 formats above). See ferrox_quant’s module docs for the block layout and independent Python cross-validation.

Q5K

Q6K

Q2K

The two more-aggressive K-quant tiers, used in Q2_K/Q3_K_M/ Q3_K_L-style quant mixes (the far more common Q4_K_M/Q5_K_M mixes only combine with Q6_K, already covered above). See ferrox_quant’s module docs and independent Python cross-validation.

Q3K

Q4_1

Legacy, largely-obsolete-for-new-releases formats, still occasionally encountered. See ferrox_quant’s module docs; byte layouts verified against real ggml-common.h source.

Q5_0

Q5_1

Q8_1

IQ4NL

Non-linear (“codebook”) quants: a 4-bit index maps through a shared 16-entry signed lookup table instead of a linear nibble*scale+min transform. See ferrox_quant’s module docs and independent Python cross-validation.

IQ4XS

IQ1S

The codebook-grid low-bit formats used throughout published “Dynamic” low-bit GGUFs of large MoE models (grid-table magnitudes + shared sign patterns; scalar kernels only so far). See ferrox_quant’s module docs and the ggml-cross-validated independent Python reference.

IQ2XXS

IQ3XXS

IQ2XS

The second codebook-grid tier (ggml tags 17/21/22/29), which the published UD-* recipes reach for when the _XXS tier is too lossy – IQ3_S especially, since it is most of what an IQ3_M mix contains. Scalar kernels only; goldens are the real compiled ggml dequantizers’ own output, asserted bit-exactly.

IQ2S

IQ3S

IQ1M

Mxfp4Gguf

GGUF block-MXFP4 (17-byte interleaved blocks, ggml tag 39) – not the same layout as WeightMatrix::Mxfp4’s two-buffer safetensors form, though the math is identical. Scalar kernel only so far.

Implementations

impl QuantKind

pub const ALL: &'static [QuantKind]

Every variant, so exhaustiveness can be tested rather than trusted. The kernel-coverage tests below iterate this; adding a variant without adding it here fails to compile (the match in Self::name is exhaustive and this list is checked against it).

pub fn name(self) -> &'static str

The GGUF-facing name. Also the key [ferrox_metal::gpu::matvec_launch_meta] is looked up by, which is why it is one function and not a Debug impl.

Trait Implementations

impl Clone for QuantKind

fn clone(&self) -> QuantKind

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for QuantKind

impl Debug for QuantKind

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for QuantKind

impl Hash for QuantKind

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl PartialEq for QuantKind

fn eq(&self, other: &QuantKind) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for QuantKind