Struct GatherQmm 

#[non_exhaustive]
pub struct GatherQmm<'a> {
    pub transpose: bool,
    pub sorted_indices: bool,
    pub global_scale: Option<&'a Array>,
}

Options of gather_qmm, built with setters. GatherQmm::default() multiplies by the transposed weights (MLX’s default).

Fields (Non-exhaustive)

Non-exhaustive structs could have additional fields added in future. Therefore, non-exhaustive structs cannot be constructed in external crates using the traditional Struct { .. } syntax; cannot be matched against without a wildcard ..; and struct update syntax will not work.

transpose: bool

x · wᵀ (MLX’s default) rather than x · w.

sorted_indices: bool

The right indices are sorted (rows grouped by expert), so one kernel can walk each expert’s weights once.

global_scale: Option<&'a Array>

nvfp4 only: one float32 scale per expert (w.weight’s batch shape), multiplying the expert’s decoded weights by global_scale / (448 · 6). Needs the right indices.

Implementations

impl<'a> GatherQmm<'a>

pub fn groups(rows: usize, experts: usize) -> bool

Whether a gather of rows sorted index rows over experts weight batches takes the grouped kernel (each expert’s rows in tiles), rather than one product per row. Two gathers that take the same kernel give a row the same bits, whatever other rows are gathered with it: a caller batching gathers keeps each one’s bits when its rows and the batch’s agree here.

pub fn transpose(self, transpose: bool) -> Self

Whether the weights are used transposed.

pub fn sorted_indices(self, sorted_indices: bool) -> Self

Whether the right indices are sorted.

pub fn global_scale(self, global_scale: &'a Array) -> Self

The per-expert nvfp4 global scale.

Trait Implementations

impl<'a> Clone for GatherQmm<'a>

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<'a> Copy for GatherQmm<'a>

impl<'a> Debug for GatherQmm<'a>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for GatherQmm<'_>

fn default() -> Self

Returns the “default value” for a type.