Struct LoRALayer 

pub struct LoRALayer { /* private fields */ }

LoRA layer: adds trainable low-rank adaptation to a frozen base weight

Implementations

impl LoRALayer

pub fn new( base_weight: Tensor, d_out: usize, d_in: usize, rank: usize, alpha: f32, ) -> Self

Create a new LoRA layer

Arguments

• base_weight - Frozen pretrained weight [d_out * d_in]
• d_out - Output dimension
• d_in - Input dimension
• rank - LoRA rank (typically 4, 8, 16, 32, or 64)
• alpha - LoRA scaling parameter (often same as rank)

Returns

LoRA layer with randomly initialized A (Gaussian) and zero-initialized B

pub fn new_with_scaling( base_weight: Tensor, d_out: usize, d_in: usize, rank: usize, alpha: f32, scaling: LoRAScaling, ) -> Self

Create a new LoRA layer with explicit scaling mode (ENT-LoRA-004)

Use LoRAScaling::RsLoRA for rank-stable training (recommended for rank > 16).

pub fn with_scale(self, scale: f32) -> Self

Override the LoRA scaling factor.

Used when restoring a serialized adapter whose scale was produced by a non-Standard mode (e.g. rsLoRA, where scale = alpha / sqrt(rank) rather than the alpha / rank that LoRALayer::new recomputes). The adapter stores the resulting scale value, not the scaling mode, so restoration sets it directly.

pub fn with_dropout(self, p: f32) -> Self

Set the LoRA dropout probability (PMAT-879).

Matches HF PEFT lora_dropout: dropout is applied to the input x before the down-projection A. p is clamped to [0.0, 1.0); 0.0 disables dropout. Dropout is only active in training mode (see LoRALayer::train).

pub fn with_dropout_seed(self, seed: u64) -> Self

Set the deterministic dropout RNG seed (PMAT-879).

With a fixed seed, dropout masks are fully reproducible, which makes the training-mode forward path testable.

pub fn train(&mut self)

Switch the layer to training mode. Dropout is active when dropout > 0.0.

pub fn eval(&mut self)

Switch the layer to evaluation mode (the default). Dropout is the identity, so inference output is unchanged — matching PEFT nn.Dropout in eval.

pub fn set_training(&mut self, training: bool)

Set training/eval mode explicitly.

pub fn is_training(&self) -> bool

Whether the layer is in training mode.

pub fn dropout(&self) -> f32

LoRA dropout probability.

pub fn forward(&self, x: &Tensor) -> Tensor

Forward pass: y = W@x + scale * (B @ (A @ dropout(x)))

Arguments

• x - Input tensor [d_in]

Returns

Output tensor [d_out]

pub fn merge(&mut self)

Merge LoRA weights into base weight: W’ = W + scale * (B @ A)

After merging, forward pass only uses W’ (more efficient). This is typically done for inference.

pub fn unmerge(&mut self)

Unmerge LoRA weights from base weight: W = W’ - scale * (B @ A)

Reverses the merge operation. Useful for continuing training or switching adapters.

pub fn base_weight(&self) -> &Tensor

Get reference to base weight matrix

pub fn lora_a(&self) -> &Tensor

Get reference to LoRA A matrix

pub fn lora_a_mut(&mut self) -> &mut Tensor

Get mutable reference to LoRA A matrix

pub fn lora_b(&self) -> &Tensor

Get reference to LoRA B matrix

pub fn lora_b_mut(&mut self) -> &mut Tensor

Get mutable reference to LoRA B matrix

pub fn trainable_params(&mut self) -> Vec<&mut Tensor>

Get trainable parameters (A and B)

pub fn is_merged(&self) -> bool

Check if LoRA is merged

pub fn rank(&self) -> usize

Get rank

pub fn scale(&self) -> f32

Get scale factor

pub fn d_out(&self) -> usize

Get output dimension

pub fn d_in(&self) -> usize

Get input dimension

Trait Implementations

impl Clone for LoRALayer

fn clone(&self) -> LoRALayer

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.