Struct QLoraLinear 

pub struct QLoraLinear { /* private fields */ }

QLoRA: Quantized LoRA for memory-efficient fine-tuning.

The base weight matrix is quantized to 4-bit NF4 format with group-wise quantization, while LoRA matrices A and B remain in full fp32 precision. This dramatically reduces memory usage for the frozen base weights.

Implementations

impl QLoraLinear

pub fn from_weight( weight: Array2<f32>, rank: usize, alpha: f32, group_size: usize, ) -> ModelResult<Self>

Create a QLoRA layer from a full-precision weight matrix.

The weight is quantized to 4-bit NF4 format with the given group size. LoRA matrices are initialized as in standard LoRA (A=Kaiming, B=zeros).

pub fn dequantize_weight(&self) -> ModelResult<Array2<f32>>

Dequantize the weight matrix back to full precision.

This is an approximate reconstruction — quantization is lossy.

pub fn forward(&self, input: &Array1<f32>) -> ModelResult<Array1<f32>>

Forward pass: dequantize weight, compute W @ x + scaling * B @ (A @ x)

pub fn memory_saved_bytes(&self) -> usize

Memory saved compared to storing full fp32 weights, in bytes.

pub fn trainable_params(&self) -> usize

Number of trainable parameters (LoRA A and B)

pub fn lora_a(&self) -> &Array2<f32>

Get the LoRA A matrix

pub fn lora_b(&self) -> &Array2<f32>

Get the LoRA B matrix

pub fn group_size(&self) -> usize

Get the quantization group size

pub fn rank(&self) -> usize

Get the rank

pub fn out_features(&self) -> usize

Get output features

pub fn in_features(&self) -> usize

Get input features

pub fn alpha(&self) -> f32

Get the alpha scaling factor

pub fn zero_point(&self) -> &Array1<f32>

Get the per-group zero points

pub fn scale(&self) -> &Array1<f32>

Get the per-group scales

Trait Implementations

impl Clone for QLoraLinear

fn clone(&self) -> QLoraLinear

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for QLoraLinear

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.