Enum QuantType 

pub enum QuantType {
    Q4_0,
    Q4_1,
    Q5_0,
    Q5_1,
    Q8_0,
    Q8_1,
    Q2K,
    Q3K,
    Q4_K_M,
    Q5K,
    Q6K,
}

Quantization block format for the QMatMul fused op (crate::registry::FusedOpParams::QMatMul). Matches the GGML/GGUF block layouts; each variant implies a fixed bytes-per-block and elements-per-block. Only variants for which a backend has a fused dequant-in-kernel matmul are currently exposed.

Variants

Q4_0

32-element block = 2 bytes f16 scale + 16 bytes packed u4 quants.

Q4_1

32-element block = 2 bytes f16 d + 2 bytes f16 m + 16 bytes packed u4 quants.

Q5_0

32-element block = 2 bytes f16 d + 4 bytes high-bit + 16 bytes packed u4 quants.

Q5_1

32-element block = 2 bytes f16 d + 2 bytes f16 m + 4 bytes high-bit + 16 bytes packed u4.

Q8_0

32-element block = 2 bytes f16 scale + 32 bytes i8 quants.

Q8_1

32-element block = 2 bytes f16 d + 2 bytes f16 s + 32 bytes i8 quants.

Q2K

256-element super-block = scales + 64-byte 2-bit packed quants + 4 bytes f16 d/dmin.

Q3K

256-element super-block = hmask + 64-byte 2-bit packed quants + 12-byte scales + 2 bytes f16 d.

Q4_K_M

256-element super-block = 2 bytes f16 d + 2 bytes f16 dmin + 12 bytes of 6-bit-packed sub-block scales/mins + 128 bytes of 4-bit-packed quants. GGML k-quant “medium” format.

Q5K

256-element super-block = d + dmin + 12-byte scales + 32-byte hmask + 128-byte qs.

Q6K

256-element super-block = 128-byte ql + 64-byte qh + 16-byte i8 scales + 2 bytes f16 d.

Implementations

impl QuantType

pub fn bytes_per_block(self) -> usize

Bytes per quantization block. Mirrors std::mem::size_of::<BlockQ*>().

pub fn elements_per_block(self) -> usize

Elements per quantization block.

Trait Implementations

impl Clone for QuantType

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for QuantType

impl Debug for QuantType

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for QuantType

impl Hash for QuantType

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl PartialEq for QuantType

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for QuantType