Struct LoraLayer 

pub struct LoraLayer {
    pub weight_a: Vec<Vec<f64>>,
    pub weight_b: Vec<Vec<f64>>,
    pub base_weight: Vec<Vec<f64>>,
    pub config: LoraConfig,
    pub merged: bool,
    /* private fields */
}

A single LoRA-augmented weight matrix.

Holds the frozen base weight W (d x k) and the low-rank factors B (d x r, init zeros) and A (r x k, init Gaussian). Forward: output = input @ (W + scaling * B @ A)^T.

Fields

weight_a: Vec<Vec<f64>>

A matrix (r x k), initialised with random Gaussian N(0, 1/r).

weight_b: Vec<Vec<f64>>

B matrix (d x r), initialised to zeros.

base_weight: Vec<Vec<f64>>

Original frozen weight (d x k).

config: LoraConfig

Configuration.

merged: bool

Whether delta W has been merged into base_weight.

Implementations

impl LoraLayer

pub fn new(base_weight: Vec<Vec<f64>>, config: LoraConfig) -> LoraResult<Self>

Create a new LoRA layer wrapping base_weight (d x k).

weight_a is initialised from N(0, 1/r) and weight_b from zeros, so the initial delta W is the zero matrix.

pub fn effective_weight(&self) -> LoraResult<Vec<Vec<f64>>>

Compute the effective weight W + scaling * B @ A without mutating state. When merged, returns base_weight (the delta is already folded in).

pub fn forward(&mut self, input: &[Vec<f64>]) -> LoraResult<Vec<Vec<f64>>>

Forward pass: output = input @ effective_weight^T.

input has shape (n, k) and the result has shape (n, d). When not merged, applies optional dropout on the LoRA branch.

pub fn merge(&mut self) -> LoraResult<()>

Merge scaling * B @ A into base_weight.

pub fn unmerge(&mut self) -> LoraResult<()>

Remove scaling * B @ A from base_weight.

pub fn trainable_params(&self) -> usize

Number of trainable parameters: r * (d + k).

pub fn total_params(&self) -> usize

Total parameter count: d * k + r * (d + k).

pub fn compression_ratio(&self) -> f64

Fraction of trainable vs total parameters.