Struct LoRAProjection 

pub struct LoRAProjection {
    pub base_weight: Tensor,
    pub lora_a: Tensor,
    pub lora_b: Tensor,
    pub d_in: usize,
    pub d_out: usize,
    pub rank: usize,
    pub scale: f32,
}

LoRA-enabled linear projection

Computes: y = x @ W + scale * (x @ A) @ B Where W is frozen base weight, A and B are trainable LoRA adapters

Fields

base_weight: Tensor

Base weight (frozen), shape (d_in × d_out)

lora_a: Tensor

LoRA A matrix (down-projection), shape (d_in × rank)

lora_b: Tensor

LoRA B matrix (up-projection), shape (rank × d_out)

d_in: usize

Input dimension

d_out: usize

Output dimension

rank: usize

LoRA rank

scale: f32

Scaling factor (alpha / rank)

Implementations

impl LoRAProjection

pub fn new( base_weight: Tensor, d_in: usize, d_out: usize, rank: usize, alpha: f32, ) -> Self

Create a new LoRA projection

Arguments

• base_weight - Frozen base weight [d_in × d_out]
• d_in - Input dimension
• d_out - Output dimension
• rank - LoRA rank (typically 4, 8, 16, 32, or 64)
• alpha - LoRA scaling parameter

pub fn forward(&self, x: &Tensor, seq_len: usize) -> Tensor

Forward pass with LoRA

Computes: y = x @ W + scale * (x @ A) @ B

Arguments

• x - Input tensor [seq_len × d_in]
• seq_len - Sequence length

Returns

Output tensor [seq_len × d_out]

pub fn lora_params(&self) -> Vec<&Tensor>

Get trainable LoRA parameters

pub fn lora_params_mut(&mut self) -> Vec<&mut Tensor>

Get mutable trainable LoRA parameters