Struct LoraLinear 

pub struct LoraLinear {
    pub base: LoraBase,
    pub lora_a: Array,
    pub lora_b: Array,
    pub scale: f32,
}

A frozen linear layer plus a trainable low-rank update (mlx-lm’s LoRALinear): y = base(x) + scale · (x A) B, with A of shape [input, r] and B of shape [r, output] (zeros at first, so the layer starts as its base).

Fields

base: LoraBase

The adapted layer (frozen).

lora_a: Array

[input_dims, r].

lora_b: Array

[r, output_dims].

scale: f32

The update’s scale (mlx-lm’s default 20).

Implementations

impl LoraLinear

pub fn from_base(base: LoraBase, r: usize, scale: f32) -> Result<Self>

Adapts base with rank r, as mlx-lm’s LoRALinear.from_base (including its random draws: A is uniform in [-1/√input, 1/√input)).

pub fn forward(&self, x: &Array) -> Result<Array>

base(x) + scale · (x A) B, the update cast to x’s dtype.

pub fn fuse(&self) -> Result<Linear>

The base layer with the update merged into its weight (mlx-lm’s fuse with dequantize): W + scale · Bᵀ Aᵀ.

Trait Implementations

impl Clone for LoraLinear

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for LoraLinear

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Module for LoraLinear

fn visit(&mut self, p: &mut Parameters<'_>)

Reports every parameter and submodule to p, frozen ones included.

fn parameters(&self) -> Vec<(String, Array)>

Every parameter with its path (layers.0.weight), in visiting order.

fn trainable_parameters(&self) -> Vec<(String, Array)>

The parameters that are trained (not frozen), with their paths.

fn update_trainable(&mut self, values: &[Array]) -> Result<()>

Replaces the trainable parameters, given in the order of Module::trainable_parameters (as optimizers return them).

fn load_weights( &mut self, weights: impl IntoIterator<Item = (String, Array)>, strict: bool, ) -> Result<()>

Replaces parameters by path, as MLX’s load_weights: each weight must have its parameter’s shape (the dtype is kept as loaded). With strict, every parameter must be given and no unknown names are allowed.