Struct QuantizedLinear 

pub struct QuantizedLinear {
    pub weight: Array,
    pub scales: Array,
    pub biases: Array,
    pub bias: Option<Array>,
    pub group_size: usize,
    pub bits: u32,
}

A Linear layer with its weight quantized (MLX’s nn.QuantizedLinear): x Wᵀ + b computed by quantized_matmul from the packed weight, its scales and biases.

The packed weight, scales and biases are frozen; the bias stays trainable, as in MLX.

Fields

weight: Array

The packed weight (uint32), [output_dims, input_dims · bits / 32].

scales: Array

One scale per group of input values, [output_dims, input_dims / group_size].

biases: Array

One bias per group of input values, shaped like scales.

bias: Option<Array>

The layer’s bias, [output_dims].

group_size: usize

Input values per scale and bias.

bits: u32

Bits per weight.

Implementations

impl QuantizedLinear

pub fn from_linear( linear: &Linear, group_size: usize, bits: u32, ) -> Result<Self>

Quantizes linear’s weight in groups of group_size (32, 64 or 128) input values with bits (1–6 or 8) bits each; the bias is kept. MLX’s defaults are 64 and 4.

pub fn dequantized_weight(&self) -> Result<Array>

The dequantized weight, [output_dims, input_dims].

pub fn forward(&self, x: &Array) -> Result<Array>

x Wᵀ + b over the last axis of x.

Trait Implementations

impl Clone for QuantizedLinear

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for QuantizedLinear

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Module for QuantizedLinear

fn visit(&mut self, p: &mut Parameters<'_>)

Reports every parameter and submodule to p, frozen ones included.

fn parameters(&self) -> Vec<(String, Array)>

Every parameter with its path (layers.0.weight), in visiting order.

fn trainable_parameters(&self) -> Vec<(String, Array)>

The parameters that are trained (not frozen), with their paths.

fn update_trainable(&mut self, values: &[Array]) -> Result<()>

Replaces the trainable parameters, given in the order of Module::trainable_parameters (as optimizers return them).

fn load_weights( &mut self, weights: impl IntoIterator<Item = (String, Array)>, strict: bool, ) -> Result<()>

Replaces parameters by path, as MLX’s load_weights: each weight must have its parameter’s shape (the dtype is kept as loaded). With strict, every parameter must be given and no unknown names are allowed.