Struct QLoraConfig 

pub struct QLoraConfig {
    pub lora: LoraConfig,
    pub quantization: QuantizationConfig,
    pub target_modules: Vec<String>,
    pub cache_dequantized: bool,
}

Configuration for QLoRA training and inference.

Compute Dtype

CRITICAL: The compute_dtype field controls numerical precision during training.

• BF16: Required for training - 100% stability rate
• FP16: Do NOT use for training - 20% failure rate due to numerical instability
• FP32: Stable but slower, useful for debugging

Target Modules

The target_modules field controls which linear layers get LoRA adapters:

• Minimal: ["q_proj", "v_proj"] - 98% of full fine-tuning quality
• Recommended: ["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"]
  • Matches full fine-tuning quality (~99.3%)

Example

use qlora_rs::QLoraConfig;

// Use preset for best stability
let config = QLoraConfig::preset_all_bf16(64, 16);

// Or customize
let config = QLoraConfig {
    target_modules: vec!["q_proj".into(), "v_proj".into()],
    ..QLoraConfig::preset_all_bf16(32, 8)
};

Fields

lora: LoraConfig

LoRA configuration (rank, alpha, dropout).

quantization: QuantizationConfig

Quantization configuration (block size, double quant).

target_modules: Vec<String>

Target modules to apply LoRA to. Default: all linear layers in transformer blocks.

cache_dequantized: bool

Whether to cache dequantized weights (opt-in, for inference speedup). Default: false (on-the-fly dequantization saves memory).

Implementations

impl QLoraConfig

pub fn preset_all_bf16(r: usize, alpha: usize) -> QLoraConfig

Create preset targeting all linear layers with BF16 compute.

Recommended for training. Matches QLoRA paper configuration.

Arguments

• r - LoRA rank (typical: 8-64)
• alpha - LoRA scaling factor (typical: 16-32)

pub fn preset_qv_bf16(r: usize, alpha: usize) -> QLoraConfig

Create preset targeting only attention Q/V projections with BF16 compute.

Memory-optimal preset: fewer trainable parameters. Achieves ~98% of full fine-tuning quality.

pub fn preset_inference(r: usize, alpha: usize) -> QLoraConfig

Create preset for inference with weight caching enabled.

Uses cached dequantization for faster inference at cost of memory.

pub fn is_target(&self, module_name: &str) -> bool

Check if a module should have LoRA applied.

pub fn scale(&self) -> f64

Get the LoRA scaling factor (alpha / r).

pub fn validate_for_training(&self) -> Result<(), QLoraError>

Validate configuration for training.

Errors

Returns error if configuration is invalid for training.

Trait Implementations

impl Clone for QLoraConfig

fn clone(&self) -> QLoraConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for QLoraConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl Default for QLoraConfig

fn default() -> QLoraConfig

Default configuration: BF16 compute, all linear layers targeted.

Note: Default uses BF16 compute dtype for training stability.

impl<'de> Deserialize<'de> for QLoraConfig

fn deserialize<__D>( __deserializer: __D, ) -> Result<QLoraConfig, <__D as Deserializer<'de>>::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Serialize for QLoraConfig

fn serialize<__S>( &self, __serializer: __S, ) -> Result<<__S as Serializer>::Ok, <__S as Serializer>::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.