Struct LoHaLayer 

pub struct LoHaLayer { /* private fields */ }

LoHa layer implementing Low-Rank Hadamard Product adaptation.

Computes: ΔW = (A1 @ B1) ⊙ (A2 @ B2) * scaling

Where:

• A1, A2: [out_features, r]
• B1, B2: [r, in_features]
• ⊙ is element-wise (Hadamard) product

Implementations

impl LoHaLayer

pub fn new( in_features: usize, out_features: usize, config: LoHaConfig, device: &Device, ) -> Result<LoHaLayer, PeftError>

Create a new LoHa layer.

Arguments

• in_features - Input dimension
• out_features - Output dimension
• config - LoHa configuration
• device - Device to create tensors on

Errors

Returns error if configuration is invalid or tensor initialization fails.

pub fn scaling(&self) -> f64

Get the scaling factor.

pub fn rank(&self) -> usize

Get the rank.

Trait Implementations

impl Adapter for LoHaLayer

type Config = LoHaConfig

The configuration type for this adapter.

fn forward( &self, input: &Tensor, base_output: Option<&Tensor>, ) -> Result<Tensor, PeftError>

Forward pass applying the adapter transformation.

fn num_parameters(&self) -> usize

Get the number of trainable parameters.

fn config(&self) -> &<LoHaLayer as Adapter>::Config

Get the adapter’s configuration.

impl Mergeable for LoHaLayer

fn merge(&self, base_weight: &Tensor) -> Result<Tensor, PeftError>

Merge adapter weights into base model weights.

fn unmerge(&self, merged_weight: &Tensor) -> Result<Tensor, PeftError>

Unmerge adapter weights from merged weights.

impl Trainable for LoHaLayer

fn register_parameters( &self, _var_map: &mut VarMap, _prefix: &str, ) -> Result<(), PeftError>

Register trainable parameters with the variable map.

fn freeze(&mut self)

Freeze all adapter parameters (disable gradients).

fn unfreeze(&mut self)

Unfreeze all adapter parameters (enable gradients).

fn is_frozen(&self) -> bool

Check if the adapter is frozen.