Struct QuantizedLinear 

pub struct QuantizedLinear {
    pub weight: Array,
    pub scales: Array,
    pub biases: Array,
    pub bias: Option<Param>,
    pub group_size: i32,
    pub bits: i32,
}

A Linear layer with its weight quantized (MLX’s nn.QuantizedLinear): x Wᵀ + b computed by quantized_matmul from the packed weight, its scales and biases.

The packed weight, scales and biases are buffers (never trained, also after unfreeze); the bias is a trainable parameter.

Fields

weight: Array

The packed weight (uint32), [output_dims, input_dims · bits / 32].

scales: Array

One scale per group of input values, [output_dims, input_dims / group_size].

biases: Array

One bias per group of input values, shaped like scales.

bias: Option<Param>

The layer’s bias, [output_dims].

group_size: i32

Input values per scale and bias.

bits: i32

Bits per weight.

Implementations

impl QuantizedLinear

pub fn from_linear(linear: &Linear) -> Self

Quantizes linear’s weight with MLX’s defaults (groups of 64 values, 4 bits); the bias is kept.

pub fn from_linear_with(linear: &Linear, group_size: i32, bits: i32) -> Self

Quantizes linear’s weight in groups of group_size (32, 64 or 128) input values with bits (1–6 or 8) bits each; the bias is kept.

pub fn dequantized_weight(&self) -> Array

The dequantized weight, [output_dims, input_dims].

pub fn forward(&self, x: &Array) -> Array

x Wᵀ + b over the last axis of x.

Trait Implementations

impl Clone for QuantizedLinear

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for QuantizedLinear

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl From<QuantizedLinear> for LoraBase

fn from(q: QuantizedLinear) -> Self

Converts to this type from the input type.

impl Leaves for QuantizedLinear

fn visit(&self, __v: &mut Visit<'_>)

Reports every leaf (and child tree, enum variant, mode flag and static field) to v, in order.

impl Tree for QuantizedLinear

where Self: Clone,

fn visit_mut(&mut self, __v: &mut VisitMut<'_>)

Reports every leaf (and child tree and mode flag) to v mutably, in the order of Leaves::visit.

fn for_each(&self, f: impl FnMut(&str, &Array))

Calls f(path, leaf) for every leaf, in order.

fn map(&self, f: impl FnMut(&str, &Array) -> Array) -> Self

The tree with every leaf replaced by f(path, leaf) (same structure; parameters keep their frozen flag).

fn leaves(&self) -> Vec<Array>

The leaves, in order.

fn paths(&self) -> Vec<String>

The leaves’ paths, in order.

fn flatten(&self) -> Vec<(String, Array)>

The leaves with their paths, in order (MLX’s tree_flatten).

fn unflatten( &self, leaves: impl IntoIterator<Item = Array>, ) -> Result<Self, Error>

A tree of this structure (self is the template) with leaves in order: exactly Tree::len of them, else an error naming both counts.

fn load_flat( &mut self, named: impl IntoIterator<Item = (String, Array)>, mode: LoadMode, ) -> Result<(), Error>

Replaces leaves by path. LoadMode::Strict requires exactly the tree’s paths with matching shapes; LoadMode::Partial assigns the matching names and ignores unknown ones (shapes must still match). Loaded dtypes are kept. On any problem the error lists every one (missing, unknown and mismatched names, sorted) and the tree is unchanged. Duplicate paths (in the tree or in named) are an error.

fn len(&self) -> usize

The number of leaves.

fn is_empty(&self) -> bool

Whether the tree has no leaves.

fn def(&self) -> TreeDef

The tree’s structure.