Enum DecomposedQuantMethod 

pub enum DecomposedQuantMethod {
    Awq {
        group_size: usize,
    },
    Gptq {
        group_size: usize,
    },
}

Quantization method for decomposed (non-monolithic) formats.

Unlike GGUF’s block-quantized QuantTensor, AWQ and GPTQ store weights as separate component tensors (qweight, scales, qzeros). This enum identifies the specific format so the correct dequantization formula and packing layout are used.

Variants

Awq

AWQ (Activation-aware Weight Quantization)

Packing: 8 INT4 values per u32, AWQ order shifts [0,16,4,20,8,24,12,28] Dequant: w = (q - zero) * scale

Fields

group_size: usize

Gptq

GPTQ (Generative Pre-trained Transformer Quantization)

Packing: 8 INT4 values per u32, sequential 4-bit packing Dequant: w = q * scale + zero Additional g_idx tensor for column permutation

Fields

group_size: usize

Implementations

impl DecomposedQuantMethod

pub fn group_size(&self) -> usize

Group size for this method

Trait Implementations

impl Clone for DecomposedQuantMethod

fn clone(&self) -> DecomposedQuantMethod

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for DecomposedQuantMethod

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for DecomposedQuantMethod

fn eq(&self, other: &DecomposedQuantMethod) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Copy for DecomposedQuantMethod

impl Eq for DecomposedQuantMethod

impl StructuralPartialEq for DecomposedQuantMethod