Struct QLoRALayer 

pub struct QLoRALayer { /* private fields */ }

QLoRA layer with 4-bit quantized base weight

Memory-efficient variant of LoRALayer that stores the frozen base weight in 4-bit quantized format, reducing memory usage by ~75%. Optionally uses double quantization (ENT-LoRA-008) for additional savings.

Implementations

impl QLoRALayer

pub fn from_lora(lora_layer: LoRALayer) -> Self

Create QLoRA layer from existing LoRA layer

Arguments

• lora_layer - Existing LoRALayer to convert

Returns

QLoRALayer with quantized base weight

pub fn from_lora_double_quant(lora_layer: LoRALayer) -> Self

Create QLoRA layer from LoRALayer with double quantization (ENT-LoRA-008)

pub fn new( base_weight: Tensor, d_out: usize, d_in: usize, rank: usize, alpha: f32, ) -> Self

Create QLoRA layer directly with quantized base weight

Arguments

• base_weight - Base weight to quantize [d_out * d_in]
• d_out - Output dimension
• d_in - Input dimension
• rank - LoRA rank
• alpha - LoRA alpha parameter

pub fn forward(&self, x: &Tensor) -> Tensor

Forward pass with on-the-fly dequantization

Arguments

• x - Input tensor [d_in]

Returns

Output tensor [d_out]

pub fn lora_a(&self) -> &Tensor

Get reference to LoRA A matrix

pub fn lora_a_mut(&mut self) -> &mut Tensor

Get mutable reference to LoRA A matrix

pub fn lora_b(&self) -> &Tensor

Get reference to LoRA B matrix

pub fn lora_b_mut(&mut self) -> &mut Tensor

Get mutable reference to LoRA B matrix

pub fn trainable_params(&mut self) -> Vec<&mut Tensor>

Get trainable parameters (A and B)

pub fn rank(&self) -> usize

Get rank

pub fn scale(&self) -> f32

Get scale factor

pub fn d_out(&self) -> usize

Get output dimension

pub fn d_in(&self) -> usize

Get input dimension

pub fn memory_stats(&self) -> MemoryStats

Get memory usage statistics

pub fn is_merged(&self) -> bool

Check if merged (always false for quantized layers)

pub fn merge_to_f32(&self) -> Vec<f32>

Merge adapter into dequantized base weight, returning full-precision f32 weights

Computes: dequantize(base_4bit) + scale * B @ A

This produces a merged weight matrix that can be exported as SafeTensors or GGUF. The result is a flat Vec of shape [d_out, d_in] in row-major layout.

pub fn base_weight_quantized(&self) -> &Quantized4Bit

Get reference to quantized base weights

pub fn is_double_quantized(&self) -> bool

Check if double quantization is enabled (ENT-LoRA-008)