Struct Quant 

pub struct Quant {
    pub mode: QuantMode,
    pub group_size: i32,
    pub bits: i32,
}

How weights are quantized: a mode, a group size and a bit width. Quant::new gives MLX’s defaults for a mode; an unsupported combination fails in the operation that uses it (a failed array, as every operation’s errors).

use nuvec_core::{Quant, QuantMode};

assert_eq!(Quant::new(QuantMode::Mxfp4), Quant { mode: QuantMode::Mxfp4, group_size: 32, bits: 4 });
assert_eq!(Quant::affine(32, 8).bits, 8);
assert_eq!(Quant::default(), Quant::affine(64, 4));

Fields

mode: QuantMode

The mode.

group_size: i32

Values per scale (affine: 32, 64 or 128; mxfp4 and mxfp8: 32; nvfp4: 16).

bits: i32

Bits per value (affine: 1–6 or 8; mxfp4 and nvfp4: 4; mxfp8: 8).

Implementations

impl Quant

pub const fn new(mode: QuantMode) -> Self

mode with MLX’s default group size and bits: affine 64/4, mxfp4 32/4, mxfp8 32/8, nvfp4 16/4.

pub const fn affine(group_size: i32, bits: i32) -> Self

Affine quantization with group_size values per scale and bias and bits per value.

pub const fn group_size(self, group_size: i32) -> Self

This quantization with group_size values per scale.

pub const fn bits(self, bits: i32) -> Self

This quantization with bits bits per value.

Trait Implementations

impl Clone for Quant

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for Quant

impl Debug for Quant

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for Quant

fn default() -> Self

Affine, 64 values per group, 4 bits.

impl Eq for Quant

impl From<QuantMode> for Quant

fn from(mode: QuantMode) -> Self

Converts to this type from the input type.

impl Hash for Quant

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl PartialEq for Quant

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl Static for Quant

fn hash_static(&self, h: &mut dyn Hasher)

Feeds the value to h.

impl StructuralPartialEq for Quant