Struct QuantizedLinear 

pub struct QuantizedLinear {
    pub weight: Array,
    pub scales: Array,
    pub biases: Option<Array>,
    pub bias: Option<Param>,
    pub quant: Quant,
}

A Linear layer with its weight quantized (MLX’s nn.QuantizedLinear): x Wᵀ + b computed by quantized_matmul from the packed weight and its scales (and, for affine, biases), in any QuantMode.

The packed weight, scales and biases are buffers (never trained, also after unfreeze); the bias is a trainable parameter.

Fields

weight: Array

The packed weight (uint32), [output_dims, input_dims · bits / 32].

scales: Array

One scale per group of input values, [output_dims, input_dims / group_size] (uint8 codes for the floating-point modes).

biases: Option<Array>

One bias per group of input values, shaped like scales: affine only.

bias: Option<Param>

The layer’s bias, [output_dims].

quant: Quant

How the weight is quantized.

Implementations

impl QuantizedLinear

pub fn from_linear(linear: &Linear) -> Self

Quantizes linear’s weight with MLX’s defaults (affine, groups of 64 values, 4 bits); the bias is kept.

pub fn from_linear_with(linear: &Linear, quant: impl Into<Quant>) -> Self

Quantizes linear’s weight with quant (a mode such as QuantMode::Mxfp4, or Quant::affine(group_size, bits)); the bias is kept.

pub fn quantized(&self) -> Quantized

The quantized weight.

pub fn dequantized_weight(&self, dtype: impl Into<Option<DType>>) -> Array

The dequantized weight, [output_dims, input_dims], in dtype (default: the scales’ dtype for affine, bfloat16 for the floating-point modes).

pub fn forward(&self, x: &Array) -> Array

x Wᵀ + b over the last axis of x.

Trait Implementations

impl Clone for QuantizedLinear

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for QuantizedLinear

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl From<QuantizedLinear> for LoraBase

fn from(q: QuantizedLinear) -> Self

Converts to this type from the input type.

impl Leaves for QuantizedLinear

fn visit(&self, __v: &mut Visit<'_>)

Reports every leaf (and child tree, enum variant, mode flag and static field) to v, in order.

impl Tree for QuantizedLinear

where Self: Clone,

fn visit_mut(&mut self, __v: &mut VisitMut<'_>)

Reports every leaf (and child tree and mode flag) to v mutably, in the order of Leaves::visit.

fn for_each(&self, f: impl FnMut(&str, &Array))

Calls f(path, leaf) for every leaf, in order.

fn map(&self, f: impl FnMut(&str, &Array) -> Array) -> Self

The tree with every leaf replaced by f(path, leaf) (same structure; parameters keep their frozen flag).

fn leaves(&self) -> Vec<Array>

The leaves, in order.

fn paths(&self) -> Vec<String>

The leaves’ paths, in order.

fn flatten(&self) -> Vec<(String, Array)>

The leaves with their paths, in order (MLX’s tree_flatten).

fn unflatten( &self, leaves: impl IntoIterator<Item = Array>, ) -> Result<Self, Error>

A tree of this structure (self is the template) with leaves in order: exactly Tree::len of them, else an error naming both counts.

fn load_flat( &mut self, named: impl IntoIterator<Item = (String, Array)>, mode: LoadMode, ) -> Result<(), Error>

Replaces leaves by path. LoadMode::Strict requires exactly the tree’s paths with matching shapes; LoadMode::Partial assigns the matching names and ignores unknown ones (shapes must still match). Loaded dtypes are kept. On any problem the error lists every one (missing, unknown and mismatched names, sorted) and the tree is unchanged. Duplicate paths (in the tree or in named) are an error.

fn len(&self) -> usize

The number of leaves.

fn is_empty(&self) -> bool

Whether the tree has no leaves.

fn def(&self) -> TreeDef

The tree’s structure.