Enum QuantKind 

pub enum QuantKind {
    Gptq {
        bits: u32,
        group_size: usize,
        desc_act: bool,
    },
    Awq {
        bits: u32,
        group_size: usize,
    },
    Gguf {
        quant_type: GgufQuantType,
    },
}

Quantization flavour discriminator for Backend::gemm_quant.

Distinct schemes need distinct kernels. Carried as a parameter so the Backend trait does not explode with one method per quantization type.

Variants

Gptq

GPTQ: group-wise int4/int8 with scales + zeros (asymmetric) + optional g_idx.

Fields

bits: u32

group_size: usize

desc_act: bool

Awq

AWQ: activation-aware int4 with scales + zeros, different packing from GPTQ.

Fields

bits: u32

group_size: usize

Gguf

GGUF: one of k-quants / legacy quants, fully specified by the inner type.

Fields

quant_type: GgufQuantType

Trait Implementations

impl Clone for QuantKind

fn clone(&self) -> QuantKind

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for QuantKind

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.