Struct LoRALayer 

pub struct LoRALayer { /* private fields */ }

LoRA layer: adds trainable low-rank adaptation to a frozen base weight

Implementations

impl LoRALayer

pub fn new( base_weight: Tensor, d_out: usize, d_in: usize, rank: usize, alpha: f32, ) -> Self

Create a new LoRA layer

Arguments

• base_weight - Frozen pretrained weight [d_out * d_in]
• d_out - Output dimension
• d_in - Input dimension
• rank - LoRA rank (typically 4, 8, 16, 32, or 64)
• alpha - LoRA scaling parameter (often same as rank)

Returns

LoRA layer with randomly initialized A (Gaussian) and zero-initialized B

pub fn new_with_scaling( base_weight: Tensor, d_out: usize, d_in: usize, rank: usize, alpha: f32, scaling: LoRAScaling, ) -> Self

Create a new LoRA layer with explicit scaling mode (ENT-LoRA-004)

Use LoRAScaling::RsLoRA for rank-stable training (recommended for rank > 16).

pub fn forward(&self, x: &Tensor) -> Tensor

Forward pass: y = W@x + scale * (B @ (A @ x))

Arguments

• x - Input tensor [d_in]

Returns

Output tensor [d_out]

pub fn merge(&mut self)

Merge LoRA weights into base weight: W’ = W + scale * (B @ A)

After merging, forward pass only uses W’ (more efficient). This is typically done for inference.

pub fn unmerge(&mut self)

Unmerge LoRA weights from base weight: W = W’ - scale * (B @ A)

Reverses the merge operation. Useful for continuing training or switching adapters.

pub fn base_weight(&self) -> &Tensor

Get reference to base weight matrix

pub fn lora_a(&self) -> &Tensor

Get reference to LoRA A matrix

pub fn lora_a_mut(&mut self) -> &mut Tensor

Get mutable reference to LoRA A matrix

pub fn lora_b(&self) -> &Tensor

Get reference to LoRA B matrix

pub fn lora_b_mut(&mut self) -> &mut Tensor

Get mutable reference to LoRA B matrix

pub fn trainable_params(&mut self) -> Vec<&mut Tensor>

Get trainable parameters (A and B)

pub fn is_merged(&self) -> bool

Check if LoRA is merged

pub fn rank(&self) -> usize

Get rank

pub fn scale(&self) -> f32

Get scale factor

pub fn d_out(&self) -> usize

Get output dimension

pub fn d_in(&self) -> usize

Get input dimension

Trait Implementations

impl Clone for LoRALayer

fn clone(&self) -> LoRALayer

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.