Struct AdaLoraLayer 

pub struct AdaLoraLayer { /* private fields */ }

AdaLoRA layer using SVD-based parameterization.

Uses W = W0 + P * Λ * Q where:

• P: Left singular vectors (out_features × r)
• Λ: Diagonal singular values (r)
• Q: Right singular vectors (r × in_features)

This allows for dynamic rank allocation by zeroing out singular values.

Implementations

impl AdaLoraLayer

pub fn new( in_features: usize, out_features: usize, config: AdaLoraConfig, device: &Device, ) -> Result<AdaLoraLayer, PeftError>

Create a new AdaLoRA layer.

Arguments

• in_features - Input dimension
• out_features - Output dimension
• config - AdaLoRA configuration
• device - Device to create tensors on

Errors

Returns error if configuration is invalid or tensor initialization fails.

pub fn current_rank(&self) -> usize

Get the current active rank.

pub fn target_rank(&self) -> usize

Get the target rank.

pub fn init_rank(&self) -> usize

Get the initial rank.

pub fn scaling(&self) -> f64

Get the scaling factor.

pub fn update_rank_mask( &mut self, importance_scores: &Tensor, budget: usize, ) -> Result<(), PeftError>

Update the rank mask based on importance scores.

Arguments

• importance_scores - Importance score for each rank [init_r]
• budget - Number of ranks to keep

Errors

Returns error if tensor operations fail.

pub fn orthogonal_regularization(&self) -> Result<Tensor, PeftError>

Compute the orthogonal regularization loss.

Encourages P^T P ≈ I and Q Q^T ≈ I.

Errors

Returns error if tensor operations fail.

pub fn get_importance_scores(&self) -> Result<Tensor, PeftError>

Get the importance scores for rank allocation.

The importance is based on the magnitude of singular values.

Errors

Returns error if tensor operations fail.

Trait Implementations

impl Adapter for AdaLoraLayer

type Config = AdaLoraConfig

The configuration type for this adapter.

fn forward( &self, input: &Tensor, base_output: Option<&Tensor>, ) -> Result<Tensor, PeftError>

Forward pass applying the adapter transformation.

fn num_parameters(&self) -> usize

Get the number of trainable parameters.

fn config(&self) -> &<AdaLoraLayer as Adapter>::Config

Get the adapter’s configuration.

impl Mergeable for AdaLoraLayer

fn merge(&self, base_weight: &Tensor) -> Result<Tensor, PeftError>

Merge adapter weights into base model weights.

fn unmerge(&self, merged_weight: &Tensor) -> Result<Tensor, PeftError>

Unmerge adapter weights from merged weights.

impl Trainable for AdaLoraLayer

fn register_parameters( &self, _var_map: &mut VarMap, _prefix: &str, ) -> Result<(), PeftError>

Register trainable parameters with the variable map.

fn freeze(&mut self)

Freeze all adapter parameters (disable gradients).

fn unfreeze(&mut self)

Unfreeze all adapter parameters (enable gradients).

fn is_frozen(&self) -> bool

Check if the adapter is frozen.