Struct LoraLinear 

pub struct LoraLinear {
    pub base: LoraBase,
    pub lora_a: Param,
    pub lora_b: Param,
    pub scale: f32,
    pub dropout: Dropout,
}

A frozen linear layer plus a trainable low-rank update (mlx-lm’s LoRALinear): y = base(x) + scale · (dropout(x) A) B, with A of shape [input, r] and B of shape [r, output] (zeros at first, so the layer starts as its base). Names follow mlx-lm: linear.weight, lora_a, lora_b.

Fields

base: LoraBase

The adapted layer (frozen).

lora_a: Param

[input_dims, r].

lora_b: Param

[r, output_dims].

scale: f32

The update’s scale (mlx-lm’s default 20).

dropout: Dropout

Dropout on the update’s input (mlx-lm’s LoRA dropout; 0 by default).

Implementations

impl LoraLinear

pub fn from_base(base: impl Into<LoraBase>, r: i32) -> Self

Adapts base with rank r and mlx-lm’s default scale 20, as mlx-lm’s LoRALinear.from_base (including its random draws: A is uniform in [-1/√input, 1/√input)). The base is frozen.

pub fn scale(self, scale: f32) -> Self

Sets the update’s scale.

pub fn dropout(self, p: f32) -> Self

Drops the update’s input with probability p in training mode (mlx-lm’s LoRA dropout; the base sees the input unchanged).

pub fn rslora(self) -> Self

Rank-stabilised scaling (rsLoRA): treating scale as α/r, the update is scaled by α/√r = scale · √r instead.

pub fn forward(&self, x: &Array) -> Array

base(x) + scale · (dropout(x) A) B, the update cast to x’s dtype.

pub fn fuse(&self) -> Linear

The base layer with the update merged into its weight (mlx-lm’s fuse with dequantize): W + scale · Bᵀ Aᵀ.

Trait Implementations

impl Clone for LoraLinear

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for LoraLinear

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Leaves for LoraLinear

fn visit(&self, __v: &mut Visit<'_>)

Reports every leaf (and child tree, enum variant, mode flag and static field) to v, in order.

impl Tree for LoraLinear

where Self: Clone,

fn visit_mut(&mut self, __v: &mut VisitMut<'_>)

Reports every leaf (and child tree and mode flag) to v mutably, in the order of Leaves::visit.

fn for_each(&self, f: impl FnMut(&str, &Array))

Calls f(path, leaf) for every leaf, in order.

fn map(&self, f: impl FnMut(&str, &Array) -> Array) -> Self

The tree with every leaf replaced by f(path, leaf) (same structure; parameters keep their frozen flag).

fn leaves(&self) -> Vec<Array>

The leaves, in order.

fn paths(&self) -> Vec<String>

The leaves’ paths, in order.

fn flatten(&self) -> Vec<(String, Array)>

The leaves with their paths, in order (MLX’s tree_flatten).

fn unflatten( &self, leaves: impl IntoIterator<Item = Array>, ) -> Result<Self, Error>

A tree of this structure (self is the template) with leaves in order: exactly Tree::len of them, else an error naming both counts.

fn load_flat( &mut self, named: impl IntoIterator<Item = (String, Array)>, mode: LoadMode, ) -> Result<(), Error>

Replaces leaves by path. LoadMode::Strict requires exactly the tree’s paths with matching shapes; LoadMode::Partial assigns the matching names and ignores unknown ones (shapes must still match). Loaded dtypes are kept. On any problem the error lists every one (missing, unknown and mismatched names, sorted) and the tree is unchanged. Duplicate paths (in the tree or in named) are an error.

fn len(&self) -> usize

The number of leaves.

fn is_empty(&self) -> bool

Whether the tree has no leaves.

fn def(&self) -> TreeDef

The tree’s structure.