Struct Lora 

pub struct Lora {
    pub rank: usize,
    pub alpha: f64,
    pub init_std: Option<f64>,
    pub dtype: Option<FloatDType>,
    pub param_group: ParamGroup,
}

A Reparameterizer that attaches LoRA adapters to 2-D weight parameters.

It is applied via Module::apply_lora.

All existing floating-point parameters are frozen. Matching rank-2 parameters receive trainable LoRA adapters; other parameters remain frozen without adapters. No model or layer code needs to change—the same Linear (and any other module) keeps working, now producing base + scale * (a @ b) for adapted weights.

Module-owned control flags are left unchanged. Apply Module::freeze before LoRA if dropout, batch normalization running statistics and other training behavior in the base module should also be disabled.

Training state

Module::unfreeze and set_require_grad(true) deliberately enable every dense parameter, including LoRA bases, and therefore switch to full fine-tuning. To enable everything except the LoRA bases, retain a ParamGroup containing their exact paths or IDs and call unfreeze_group(ParamGroup::all().exclude(lora_base_group)).

use burn_core::module::{Lora, Module, ParamGroup};

let lora_bases = ParamGroup::from_paths(vec![
    "encoder.weight",
    "decoder.weight",
]);

model
    .apply_lora(Lora::new(2, 4.0).set_param_group(lora_bases.clone()))
    .unfreeze_group(ParamGroup::all().exclude(lora_bases))

Fields

rank: usize

Rank of the low-rank decomposition.

alpha: f64

Scaling numerator; the adapter contribution is scaled by alpha / rank.

init_std: Option<f64>

Standard deviation used to initialize the A factor. Defaults to 1 / rank.

dtype: Option<FloatDType>

Precision the factors are built at. Defaults to the base weight’s own float dtype; a quantized base has none, so a model that computes in half precision over a packed base must say so here — the factors meet its activations and gradients in elementwise ops that do not promote.

param_group: ParamGroup

The parameter group on which to apply the LoRA.

Implementations

impl Lora

pub fn new(rank: usize, alpha: f64) -> Self

Create a new LoRA reparameterizer with the given rank and alpha.

pub fn set_param_group(self, group: ParamGroup) -> Self

Set the parameter group on which to apply LoRA adapters.

pub fn set_dtype(self, dtype: FloatDType) -> Self

Set the precision the adapter factors are built at.

Trait Implementations

impl Clone for Lora

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Lora

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Reparameterizer for Lora

type Reparam = LoraAdapter

Reparameterization produced for a parameter.

fn reparameterize<const D: usize>( &mut self, path: &str, param: Param<Tensor<D>>, ) -> (Param<Tensor<D>>, Option<Self::Reparam>)

Prepare a parameter and optionally create a reparameterization for it.