Module lora

Expand description

LoRA (Low-Rank Adaptation) implementation

LoRA enables parameter-efficient fine-tuning of large pretrained models by adding trainable low-rank decomposition matrices to frozen weights.

Structs§

AdapterMetadata: Adapter metadata (without weights)
DoRALayer: DoRA layer: magnitude-direction decomposed LoRA
LoRAAdapter: Serializable LoRA adapter format
LoRAConfig: Configuration for LoRA adapter targeting
LoRALayer: LoRA layer: adds trainable low-rank adaptation to a frozen base weight
MemoryStats: Memory usage statistics for QLoRA layer
MergePublishResult: Result of merge-export-publish pipeline
MergedModel: Merged model from combining LoRA/QLoRA adapters with base weights
MultiAdapterManager: Multi-adapter manager
NamedAdapter: Named adapter wrapping a set of LoRA layers
PagedOptimStates: Paged optimizer state manager
PagedState: CPU-resident optimizer state for one parameter group
PagingStats: Paging statistics for monitoring
PeftAdapterBundle: A bundle of LoRA adapters keyed by layer path
PeftAdapterConfig: PEFT adapter configuration matching the HuggingFace PEFT schema
QLoRALayer: QLoRA layer with 4-bit quantized base weight
VramBudget: VRAM budget tracker for optimizer state paging decisions

load_adapter: Load LoRA adapter from file (Entrenar JSON format)
load_adapter_peft: Load LoRA adapter from PEFT-compatible format
merge_and_collect: Merge LoRA layers into base weights and collect as merged model
merge_export_publish: Merge LoRA adapters, export as SafeTensors, and publish to HuggingFace Hub
merge_qlora_and_collect: Merge QLoRA layers into f32 weights and collect as merged model
merge_qlora_export_publish: Merge QLoRA adapters, export as SafeTensors, and publish to HuggingFace Hub
pissa_init: Initialize a LoRA layer using PiSSA (SVD-based initialization)
save_adapter: Save LoRA adapter to file (Entrenar JSON format)
save_adapter_peft: Save LoRA adapters in PEFT-compatible format