Struct DoraLayer 

pub struct DoraLayer { /* private fields */ }

DoRA (Weight-Decomposed Low-Rank Adaptation) layer.

DoRA decomposes weight updates into magnitude and direction components: W' = m * (W + ΔW) / ||W + ΔW||

where:

• m is a learnable magnitude vector (per output dimension)
• W is the original base weight
• ΔW = B @ A * scaling is the LoRA update

Reference: https://arxiv.org/abs/2402.09353

Implementations

impl DoraLayer

pub fn new( in_features: usize, out_features: usize, config: LoraConfig, device: &Device, base_weight: Option<&Tensor>, ) -> Result<DoraLayer, PeftError>

Create a new DoRA layer.

Arguments

• in_features - Input dimension
• out_features - Output dimension
• config - LoRA configuration (with use_dora: true)
• device - Device to create tensors on
• base_weight - Optional base weight for initialization

Errors

Returns an error if the layer construction fails.

pub fn magnitude(&self) -> &Tensor

Get the magnitude vector.

pub fn lora_layer(&self) -> &LoraLayer

Get the underlying LoRA layer.

pub fn set_base_weight(&mut self, weight: Tensor)

Update the base weight reference.

Trait Implementations

impl Adapter for DoraLayer

type Config = LoraConfig

The configuration type for this adapter.

fn forward( &self, input: &Tensor, base_output: Option<&Tensor>, ) -> Result<Tensor, PeftError>

Forward pass applying the adapter transformation.

fn num_parameters(&self) -> usize

Get the number of trainable parameters.

fn config(&self) -> &<DoraLayer as Adapter>::Config

Get the adapter’s configuration.

impl Mergeable for DoraLayer

fn merge(&self, base_weight: &Tensor) -> Result<Tensor, PeftError>

Merge adapter weights into base model weights.

fn unmerge(&self, merged_weight: &Tensor) -> Result<Tensor, PeftError>

Unmerge adapter weights from merged weights.

impl Trainable for DoraLayer

fn register_parameters( &self, var_map: &mut VarMap, prefix: &str, ) -> Result<(), PeftError>

Register trainable parameters with the variable map.

fn freeze(&mut self)

Freeze all adapter parameters (disable gradients).

fn unfreeze(&mut self)

Unfreeze all adapter parameters (enable gradients).

fn is_frozen(&self) -> bool

Check if the adapter is frozen.