ruvllm 2.2.0

LLM serving runtime with Ruvector integration - Paged attention, KV cache, and SONA learning
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122

//! MicroLoRA Fine-tuning Pipeline for Real-time Per-request Adaptation
//!
//! This module provides an ultra-lightweight LoRA implementation optimized for
//! real-time adaptation with minimal overhead (<1MB per adapter).
//!
//! ## Quick Start
//!
//! ```rust,ignore
//! use ruvllm::lora::{MicroLoRA, MicroLoraConfig, TargetModule, AdaptFeedback};
//!
//! // Create MicroLoRA for hidden dimension 4096
//! let config = MicroLoraConfig::for_hidden_dim(4096);
//! let mut lora = MicroLoRA::new(config);
//!
//! // Apply LoRA during inference
//! let delta = lora.forward(&input_tensor, &TargetModule::QProj);
//! let output: Vec<f32> = base_output.iter()
//!     .zip(delta.iter())
//!     .map(|(b, d)| b + d)
//!     .collect();
//!
//! // Adapt based on quality feedback
//! let feedback = AdaptFeedback::from_quality(0.85);
//! lora.adapt(&input_tensor, feedback)?;
//! lora.apply_updates(0.01);  // learning rate
//! ```
//!
//! ## Architecture
//!
//! ```text
//! +-------------------+     +-------------------+
//! | Request           |---->| MicroLoRA         |
//! | (input tensor)    |     | - Rank 1-2        |
//! +-------------------+     | - <1ms forward    |
//!                           | - Per-request     |
//!                           +--------+----------+
//!                                    |
//!                                    v (async feedback)
//!                           +--------+----------+
//!                           | Training Pipeline |
//!                           | - EWC++ regul.    |
//!                           | - Single-example  |
//!                           | - LR scheduling   |
//!                           +--------+----------+
//!                                    |
//!                                    v
//!                           +--------+----------+
//!                           | Adapter Manager   |
//!                           | - Hot-swapping    |
//!                           | - Composition     |
//!                           | - Persistence     |
//!                           +-------------------+
//! ```
//!
//! ## Target Modules
//!
//! Choose which transformer components to adapt:
//!
//! | Module | Memory | Impact | Recommended For |
//! |--------|--------|--------|-----------------|
//! | `QProj` | Low | High | Attention focus |
//! | `KProj` | Low | Medium | Key patterns |
//! | `VProj` | Low | High | Content generation |
//! | `OProj` | Low | Medium | Output projection |
//! | `GateProj` | Medium | High | FFN routing |
//! | `UpProj` | High | Medium | FFN expansion |
//! | `DownProj` | High | Medium | FFN compression |
//!
//! ## Features
//!
//! - **Ultra-lightweight**: Rank 1-2 adapters with <1MB memory footprint
//! - **Real-time**: Per-request adaptation with <1ms forward pass
//! - **EWC++ Integration**: Prevents catastrophic forgetting during adaptation
//! - **NEON/SIMD Optimized**: Hardware-accelerated forward and backward passes
//! - **Async Adaptation**: Non-blocking training with feedback loops
//! - **Hot-swapping**: Seamlessly switch adapters without model reload
//!
//! ## Training with EWC++
//!
//! ```rust,ignore
//! use ruvllm::lora::{TrainingPipeline, TrainingConfig, EwcRegularizer};
//!
//! let config = TrainingConfig {
//!     learning_rate: 0.001,
//!     ewc_lambda: 0.1,  // Regularization strength
//!     quality_threshold: 0.5,
//!     ..Default::default()
//! };
//!
//! let mut pipeline = TrainingPipeline::new(config);
//! pipeline.init_for_lora(&lora);
//!
//! // Train on samples
//! for sample in samples {
//!     pipeline.train_step(&lora, &sample.input, sample.feedback)?;
//! }
//!
//! // Mark task boundary (computes Fisher information)
//! pipeline.start_new_task(&lora);
//! ```

pub mod adapter;
pub mod adapters;
pub mod micro_lora;
pub mod training;

// Re-exports
pub use adapter::{
    AdapterComposer, AdapterHandle, AdapterPool, AdapterRegistry, CompositionStrategy,
};
pub use adapters::{
    merge::{AdapterMerger, HotSwapManager, MergeConfig, MergeStrategy},
    trainer::{
        AdapterDataset, AdapterTrainer, AdapterTrainingConfig, SyntheticDataGenerator,
        TrainingExample,
    },
    AdapterMetadata, LoraConfig, RuvLtraAdapters,
};
pub use micro_lora::{AdaptFeedback, LoraAdapter, MicroLoRA, MicroLoraConfig, TargetModule};
pub use training::{
    EwcRegularizer, GradientAccumulator, LearningRateSchedule, TrainingConfig, TrainingPipeline,
};