Struct OftLayer 

pub struct OftLayer { /* private fields */ }

OFT layer implementing Orthogonal Fine-Tuning.

Uses block-diagonal orthogonal matrices to transform weights: W' = W @ R where R is a block-diagonal orthogonal matrix.

The orthogonal matrix R is parameterized via Cayley transform: R = (I - Q) @ (I + Q)^{-1} where Q is skew-symmetric.

Implementations

impl OftLayer

pub fn new( features: usize, config: OftConfig, device: &Device, ) -> Result<OftLayer, PeftError>

Create a new OFT layer.

Arguments

• features - Dimension of the weight matrix (must be divisible by r)
• config - OFT configuration
• device - Device to create tensors on

Errors

Returns an error if configuration validation fails or if layer construction fails.

pub fn num_blocks(&self) -> usize

Get the number of blocks.

pub fn block_size(&self) -> usize

Get the block size.

Trait Implementations

impl Adapter for OftLayer

type Config = OftConfig

The configuration type for this adapter.

fn forward( &self, input: &Tensor, base_output: Option<&Tensor>, ) -> Result<Tensor, PeftError>

Forward pass applying the adapter transformation.

fn num_parameters(&self) -> usize

Get the number of trainable parameters.

fn config(&self) -> &<OftLayer as Adapter>::Config

Get the adapter’s configuration.

impl Mergeable for OftLayer

fn merge(&self, base_weight: &Tensor) -> Result<Tensor, PeftError>

Merge adapter weights into base model weights.

fn unmerge(&self, merged_weight: &Tensor) -> Result<Tensor, PeftError>

Unmerge adapter weights from merged weights.

impl Trainable for OftLayer

fn register_parameters( &self, _var_map: &mut VarMap, _prefix: &str, ) -> Result<(), PeftError>

Register trainable parameters with the variable map.

fn freeze(&mut self)

Freeze all adapter parameters (disable gradients).

fn unfreeze(&mut self)

Unfreeze all adapter parameters (enable gradients).

fn is_frozen(&self) -> bool

Check if the adapter is frozen.