Enum QuantLinearMode 

pub enum QuantLinearMode {
    W8A8(Int8Tier),
    W8A16,
}

Which quantized linear op class the armed route runs.

W8A8 quantizes activations per row and uses the exact-i32 int8 dot — fastest, but the activation rounding perturbs logits enough that seeded sampling can draw different tokens than f32. W8A16 keeps activations f32 and dequantizes weights in-register — the same one-byte-per-weight memory traffic, no activation error, so the output tracks the f32 reference much more closely. Its f32 accumulation is lane-ordered (not the reference’s left-to-right order): this is a lossy route already, so reduction-order freedom is part of the deal, and the fidelity gate is measured downstream, not asserted bitwise.

Variants

W8A8(Int8Tier)

Int8 activations times int8 weights, exact i32 accumulation.

W8A16

f32 activations times dequantized int8 weights, lane-ordered f32 accumulation.

Implementations

impl QuantLinearMode

pub const fn as_str(self) -> &'static str

Stable machine-readable mode name.

Trait Implementations

impl Clone for QuantLinearMode

fn clone(&self) -> QuantLinearMode

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for QuantLinearMode

impl Debug for QuantLinearMode

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for QuantLinearMode

impl PartialEq for QuantLinearMode

fn eq(&self, other: &QuantLinearMode) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for QuantLinearMode