# RuvLLM
[](https://www.rust-lang.org/)
[](LICENSE)
[](#testing)
[](#architecture)
[](#huggingface-export)
[](https://www.npmjs.com/package/@ruvector/ruvllm)
[](#trm-tiny-recursive-models)
**Self-Optimizing Neural Architecture (SONA) with TRM Recursive Reasoning, LFM2 Cortex, Ruvector Memory, and Intelligent Routing**
> *"The intelligence is not in one model anymore. It is in the loop."*
---
## What is RuvLLM?
RuvLLM is a **self-learning language model orchestration system** that combines frozen foundation models with adaptive memory and intelligent routing. Unlike traditional LLMs that rely solely on static parameters, RuvLLM continuously improves from every interaction through three temporal learning loops.
**Key Innovation**: RuvLLM doesn't replace your LLM—it makes any LLM smarter over time by learning from experience, routing intelligently, and preventing catastrophic forgetting.
```
┌─────────────────────────────────────────────────────────────────────────┐
│ RuvLLM Architecture │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ Query ──► Embedding ──► Memory Search ──► Router Decision │
│ │ │ │
│ ▼ ▼ │
│ Graph Attention Model Selection │
│ │ │ │
│ └────────┬───────────┘ │
│ ▼ │
│ ┌─────────────────────┐ │
│ │ LLM Inference │ │
│ │ (Any LLM Backend) │ │
│ └─────────────────────┘ │
│ │ │
│ ▼ │
│ ┌───────────────────────────────────┐ │
│ │ SONA Learning (3 Temporal Loops) │ │
│ │ • Instant: Per-request MicroLoRA │ │
│ │ • Background: Hourly patterns │ │
│ │ • Deep: Weekly EWC++ updates │ │
│ └───────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────┘
```
---
## Features
### Core Components
| **LFM2 Cortex** | Frozen reasoning engine (135M-2.6B params) | Mock, Candle, or external (llama.cpp/vLLM) |
| **Ruvector Memory** | Adaptive synaptic mesh with HNSW indexing | Full CPU implementation with graph expansion |
| **FastGRNN Router** | Intelligent model selection circuit | Sparse + low-rank matrices with EWC learning |
| **Graph Attention** | Multi-head attention with edge features | 8-head attention, layer normalization |
| **SONA Engine** | Self-optimizing neural architecture | LoRA + EWC++ + ReasoningBank |
| **TRM Engine** | Tiny Recursive Models (7M params) | Recursive latent refinement with SONA bridge |
### TRM (Tiny Recursive Models)
RuvLLM v0.2.3 introduces **TRM** - Samsung SAIL Montreal's parameter-efficient recursive reasoning approach. TRM achieves strong reasoning performance with only **7M parameters** through iterative latent refinement.
```
┌─────────────────────────────────────────────────────────────────────────┐
│ TRM Architecture │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ Question ──┬► Latent Update (n times) ──► Answer Refine ──┐ │
│ │ │ │
│ └───────────────────────────────────────────────┘ │
│ (repeat K times) │
│ │
│ Components: │
│ • MLP Latent Updater - Fast feed-forward updates │
│ • Attention Latent Updater - Multi-head attention refinement │
│ • Confidence Scorer - Early stopping based on convergence │
│ • Answer Refiner - Residual-based answer improvement │
│ • SONA Bridge - Integration with learning loops │
│ │
└─────────────────────────────────────────────────────────────────────────┘
```
**Key Features:**
- **7M parameters** - Achieves 83% on GSM8K with minimal compute
- **Recursive refinement** - Iteratively improves answers through K iterations
- **Adaptive K** - SONA routing determines optimal iteration count
- **Early stopping** - Confidence-based termination for efficiency
- **NaN-safe** - Robust numerical guards prevent gradient explosions
- **Buffer reuse** - Optimized memory allocation for production use
### SONA: Self-Optimizing Neural Architecture
RuvLLM introduces **SONA**, a three-tier temporal learning system:
```
┌──────────────────────────────────────────────────────────────────────────┐
│ Loop A: Instant (Per-Request) Latency: <100μs │
│ ────────────────────────────────────── │
│ • Records query trajectories with activation patterns │
│ • MicroLoRA adaptation (rank 1-2) for immediate improvement │
│ • SIMD-optimized: 2,236 ops/sec throughput │
├──────────────────────────────────────────────────────────────────────────┤
│ Loop B: Background (Hourly) │
│ ───────────────────────────── │
│ • K-means++ clustering extracts patterns (100 clusters = 1.3ms search) │
│ • Base LoRA updates (rank 4-16) from successful patterns │
│ • ReasoningBank stores learned strategies │
├──────────────────────────────────────────────────────────────────────────┤
│ Loop C: Deep (Weekly) │
│ ───────────────────── │
│ • Dream consolidation across all memory │
│ • EWC++ prevents catastrophic forgetting (λ=2000 optimal) │
│ • Concept hierarchies created, old nodes archived │
└──────────────────────────────────────────────────────────────────────────┘
```
### Advanced Features
| **SIMD Inference** | Native AVX2/AVX512/SSE4.1 operations for CPU optimization |
| **Q4 Quantization** | 4-bit weight quantization for memory efficiency |
| **MicroLoRA** | Per-request adaptation with rank 1-2 (benchmark: rank-2 is 5% faster) |
| **EWC++** | Enhanced elastic weight consolidation with online Fisher estimation |
| **ReasoningBank** | Pattern storage with K-means++ clustering |
| **HuggingFace Export** | Export LoRA weights, patterns, and preference pairs |
| **Real Inference** | Candle-based inference with HuggingFace model support |
| **Multi-Model Routing** | Automatic selection between SmolLM, Qwen2, TinyLlama |
| **Federated Learning** | Distributed learning across ephemeral agents with central coordinator |
| **WASM Support** | Run SONA in browsers and edge devices |
| **Training Pipelines** | Templated training for code, chat, reasoning, and custom agents |
| **Agent Factory** | Create and manage multiple specialized learning agents |
| **TRM Reasoning** | Recursive reasoning with only 7M parameters (83% GSM8K) |
| **Adaptive K Routing** | SONA-driven iteration count for optimal compute |
| **NaN Guards** | Robust numerical stability for production deployment |
### Federated Learning Architecture
RuvLLM supports **federated learning** where ephemeral agents collect trajectories and export to a central coordinator:
```
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ Agent A │ │ Agent B │ │ Agent C │
│ (ephemeral) │ │ (ephemeral) │ │ (ephemeral) │
└──────┬──────┘ └──────┬──────┘ └──────┬──────┘
│ │ │
│ export() │ export() │ export()
▼ ▼ ▼
┌────────────────────────────────────────────────┐
│ Federated Coordinator │
│ (persistent, large capacity) │
│ • Aggregates trajectories from all agents │
│ • Quality-filtered acceptance (threshold) │
│ • Auto-consolidation every N agents │
│ • Shares patterns with new agents │
└────────────────────────────────────────────────┘
```
**Key Components**:
- **EphemeralAgent**: Short-lived agents that process tasks and export learned state
- **FederatedCoordinator**: Central aggregator with 50K trajectory capacity
- **AgentExport**: Serializable state containing trajectories, stats, and patterns
- **Quality Filtering**: Only high-quality trajectories (>0.4 score) are aggregated
---
## Performance Benchmarks
### Orchestration Latency (CPU-Only)
| **Initialization** | 3.71ms | Full system startup |
| **Average Query** | 0.09ms | Single query latency |
| **Session Query** | 0.04ms | With context reuse |
| **Throughput** | ~38,000 q/s | 8 concurrent queries |
| **Memory Footprint** | ~50MB | Base system |
### Latency Breakdown
```
Embedding: ~0.02ms ████░░░░░░ (20%)
Retrieval: ~0.01ms ██░░░░░░░░ (10%)
Routing: ~0.01ms ██░░░░░░░░ (10%)
Attention: ~0.02ms ████░░░░░░ (20%)
Generation: ~0.04ms ████████░░ (40%)
```
### SONA Learning Performance
| MicroLoRA | Throughput | 2,236 ops/sec |
| MicroLoRA | Batch-32 Latency | 0.447ms |
| ReasoningBank | Pattern Search | 1.3ms (100 clusters) |
| EWC++ | Fisher Update | <1ms |
### Comparison with Traditional Systems
| GPT-4o (API) | 450.00 | 585.00 | 1.0x (baseline) |
| Claude 3.5 Sonnet | 380.00 | 456.00 | 1.2x |
| Gemini 2.0 Flash | 180.00 | 234.00 | 2.5x |
| Llama 3.3 70B (vLLM) | 120.00 | 168.00 | 3.8x |
| **RuvLLM Orchestration** | **0.06** | **0.08** | **~7,500x** |
> **Note**: RuvLLM orchestration latency measures memory retrieval, routing, and context preparation—NOT LLM generation. Actual response quality depends on your LLM backend.
---
## Feature Comparison
| On-device Inference | ✗ | ✗ | ✗ | ✓ | ✓ |
| Continuous Learning | ✗ | ✗ | ✗ | ✗ | ✓ |
| Graph-based Memory | ✗ | ✗ | △ | ✗ | ✓ |
| Adaptive Model Routing | ✗ | ✗ | ✗ | ✗ | ✓ |
| EWC Anti-Forgetting | ✗ | ✗ | ✗ | ✗ | ✓ |
| LoRA Adaptation | ✗ | ✗ | ✗ | ✗ | ✓ |
| Pattern Extraction | ✗ | ✗ | ✗ | ✗ | ✓ |
| HuggingFace Export | ✗ | ✗ | ✗ | ✗ | ✓ |
| SIMD Optimization | ✗ | ✗ | ✗ | △ | ✓ |
| Sub-ms Orchestration | ✗ | ✗ | ✗ | ✗ | ✓ |
| Federated Learning | ✗ | ✗ | ✗ | ✗ | ✓ |
| WASM/Browser Support | ✗ | ✗ | ✗ | ✗ | ✓ |
| Training Pipelines | ✗ | ✗ | ✗ | ✗ | ✓ |
| Works with ANY LLM | ✗ | ✗ | ✓ | ✗ | ✓ |
| **TRM Recursive Reasoning** | ✗ | ✗ | ✗ | ✗ | ✓ |
| **7M Param Efficiency** | ✗ | ✗ | ✗ | ✗ | ✓ |
*Legend: ✓ = Full Support, △ = Partial, ✗ = Not Supported*
---
## Quick Start
### Prerequisites
- Rust 1.77+
- Cargo
### Installation
```bash
# Clone the repository
git clone https://github.com/ruvnet/ruvector.git
cd ruvector/examples/ruvLLM
# Build in release mode
cargo build --release
```
### Run the Demo
```bash
# Interactive demo with mock inference
cargo run --bin ruvllm-demo --release
# SIMD capabilities demo
cargo run --bin ruvllm-simd-demo --release
# Quick benchmark
cargo run --bin ruvllm-bench --release
# Full benchmark suite
cargo run --bin ruvllm-benchmark-suite --release
# HTTP server (requires 'server' feature)
cargo run --bin ruvllm-server --release --features server
# Pretraining pipeline
cargo run --bin ruvllm-pretrain --release
# HuggingFace export (requires 'hf-export' feature)
cargo run --bin ruvllm-export --release --features hf-export -- help
```
### Library Usage
```rust
use ruvllm::{Config, RuvLLM, Result};
#[tokio::main]
async fn main() -> Result<()> {
// Configure the system
let config = Config::builder()
.embedding_dim(768)
.router_hidden_dim(128)
.hnsw_params(32, 200, 64) // M, ef_construction, ef_search
.learning_enabled(true)
.build()?;
// Initialize
let llm = RuvLLM::new(config).await?;
// Create a session for multi-turn conversation
let session = llm.new_session();
// Query with session context
let response = llm.query_session(&session, "What is machine learning?").await?;
println!("Response: {}", response.text);
println!("Model: {:?}", response.routing_info.model);
println!("Confidence: {:.2}%", response.confidence * 100.0);
// Provide feedback for learning
llm.feedback(Feedback {
request_id: response.request_id,
rating: Some(5),
correction: None,
task_success: Some(true),
}).await?;
Ok(())
}
```
### SIMD Inference Engine
```rust
use ruvllm::{SimdInferenceEngine, SimdGenerationConfig, SimdOps};
// Create SIMD-optimized engine
let engine = SimdInferenceEngine::new(256, 128, 4, 4)?;
// Configure generation
let config = SimdGenerationConfig {
max_tokens: 50,
temperature: 0.7,
top_p: 0.9,
..Default::default()
};
// Generate with SIMD acceleration
let result = engine.generate("Once upon a time", &config)?;
```
### SONA Learning Loops
```rust
use ruvllm::sona::{LoopCoordinator, SonaConfig, InstantLoop, BackgroundLoop};
// Initialize SONA coordinator
let config = SonaConfig {
hidden_dim: 256,
embedding_dim: 256,
pattern_clusters: 100,
..Default::default()
};
let coordinator = LoopCoordinator::new(config);
// Instant learning (per-request)
coordinator.instant_loop().record_trajectory(query, response, quality);
// Background learning (hourly)
coordinator.background_loop().extract_patterns().await;
// Deep learning (weekly) - automatically handles EWC++
coordinator.deep_consolidation().await;
```
### TRM Recursive Reasoning
```rust
use ruvllm::trm::{TrmEngine, TrmEngineBuilder, TrmConfig, RecursiveReasoner};
// Build TRM engine with custom configuration
let mut engine = TrmEngineBuilder::new()
.hidden_dim(256)
.embedding_dim(256)
.default_k(10) // Default K iterations
.n_inner(4) // Inner latent updates per K
.confidence_threshold(0.95) // Early stopping threshold
.build()
.unwrap();
// Prepare question and answer embeddings
let question = vec![0.5; 256]; // Question embedding
let mut answer = vec![0.1; 256]; // Initial answer (refined in-place)
// Perform recursive reasoning
let result = engine.reason(&question, &mut answer);
println!("Confidence: {:.2}%", result.confidence * 100.0);
println!("Iterations used: {}/{}", result.iterations_used, result.max_iterations);
println!("Early stopped: {}", result.early_stopped);
// With SONA routing for adaptive K
use ruvllm::trm::SonaBridge;
let bridge = SonaBridge::new(256, 256);
let routing = bridge.compute_routing(&question, 0.8); // quality hint
let result = engine.reason_with_routing(&question, &mut answer, &routing);
println!("Adaptive K used: {}", routing.k);
```
### Federated Learning
```rust
use ruvector_sona::training::{EphemeralAgent, FederatedCoordinator, SonaConfig};
// Create central coordinator (persistent, large capacity)
let mut coordinator = FederatedCoordinator::default_coordinator("main", 3072);
coordinator.set_quality_threshold(0.4); // Only accept high-quality trajectories
coordinator.set_consolidation_interval(50); // Auto-consolidate every 50 agents
// Create ephemeral agents for distributed learning
let mut agent = EphemeralAgent::default_federated("agent-1", 3072);
// Agent processes tasks and learns locally
agent.process_trajectory(
embedding, // Query embedding
activations, // Hidden state activations
quality, // Quality score [0.0, 1.0]
Some("gpt-4".to_string()), // Model route
vec!["code".to_string()], // Context tags
);
// Export state before agent termination
let export = agent.export_state();
println!("Agent exported {} trajectories", export.trajectories.len());
// Coordinator aggregates learning from all agents
let result = coordinator.aggregate(export);
println!("Accepted: {}, Rejected: {}",
result.trajectories_accepted,
result.trajectories_rejected
);
// Get patterns for warm-starting new agents
let patterns = coordinator.get_initial_patterns(10);
```
### WASM Usage (Browser/Edge)
Build SONA for WebAssembly:
```bash
# Build WASM package
cd crates/sona
wasm-pack build --target web --features wasm
```
Use in JavaScript:
```javascript
import init, { WasmSonaEngine } from './pkg/sona.js';
async function main() {
await init();
// Create SONA engine
const engine = new WasmSonaEngine(256); // hidden_dim = 256
// Or with custom configuration
const engineCustom = WasmSonaEngine.withConfig({
hidden_dim: 256,
embedding_dim: 256,
micro_lora_rank: 2,
base_lora_rank: 16,
ewc_lambda: 1000.0,
pattern_clusters: 128,
});
// Start trajectory
const embedding = new Float32Array(256).fill(0.1);
const trajectoryId = engine.startTrajectory(embedding);
// Record steps
engine.recordStep(trajectoryId, 42, 0.8, 1000);
// End trajectory with quality score
engine.endTrajectory(trajectoryId, 0.85);
// Apply LoRA transformation
const input = new Float32Array(256).fill(1.0);
const output = engine.applyLora(input);
// Run learning cycles
engine.runInstantCycle(); // Flush micro-LoRA updates
if (engine.tick()) { // Background learning
console.log('Background learning completed');
}
// Get statistics
const stats = engine.stats();
console.log('Patterns:', stats.patterns_stored);
}
```
---
## HuggingFace Export
Export learned patterns, LoRA weights, and preference pairs to HuggingFace:
```bash
# Export LoRA weights in PEFT-compatible SafeTensors format
ruvllm-export safetensors ./exports/lora
# Export learned patterns as JSONL dataset
ruvllm-export patterns ./exports/patterns
# Export DPO/RLHF preference pairs
ruvllm-export preferences ./exports/preferences
# Export all artifacts
ruvllm-export all ./exports
# Push to HuggingFace Hub
HF_TOKEN=your_token ruvllm-export push username/my-sona-model
# Generate pretraining pipeline configuration
ruvllm-export pretrain ./exports
```
---
## Architecture Deep Dive
### HNSW Memory Index
The memory system uses Hierarchical Navigable Small World graphs:
```
Layer 2: [3] ─────────────────── [7]
│ │
Layer 1: [3] ─── [5] ─────────── [7] ─── [9]
│ │ │ │
Layer 0: [1]─[2]─[3]─[4]─[5]─[6]─[7]─[8]─[9]─[10]
• M = 32 connections per node
• ef_construction = 200 for build quality
• ef_search = 64 for query speed
• O(log N) search complexity
```
### FastGRNN Router
Sparse + Low-rank matrices for efficient routing:
```
Input (128-dim)
│
┌───────┴───────┐
│ LayerNorm │
└───────┬───────┘
│
┌───────────┴───────────┐
│ FastGRNN Cell │
│ │
│ W_sparse (90% zero) │
│ U = A @ B (rank-8) │
│ │
│ z = σ(Wx + Uh + b) │
│ h' = z⊙h + (1-z)⊙ν │
└───────────┬───────────┘
│
┌───────┴───────┐
│ Output Heads │
├───────────────┤
│ Model Select │ → 4 classes
│ Context Size │ → 5 buckets
│ Temperature │ → continuous
│ Top-p │ → continuous
│ Confidence │ → continuous
└───────────────┘
```
### MicroLoRA Architecture
Two-tier LoRA system for adaptive learning:
```
┌─────────────────────────────────────────────────────────────┐
│ MicroLoRA (Rank 1-2) │
│ Per-Request Adaptation │
├─────────────────────────────────────────────────────────────┤
│ │
│ Input ──► Down Proj ──► Up Proj ──► Scale ──► Add │
│ (dim) (dim→rank) (rank→dim) (α/r) to output │
│ │
│ Performance: <100μs latency, 2,236 ops/sec │
│ Rank-2 is ~5% faster than Rank-1 (better SIMD) │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ BaseLoRA (Rank 4-16) │
│ Background Adaptation │
├─────────────────────────────────────────────────────────────┤
│ │
│ Aggregated from successful MicroLoRA patterns │
│ Merged hourly into base weights │
│ EWC++ regularization prevents forgetting │
│ │
└─────────────────────────────────────────────────────────────┘
```
### EWC++ (Enhanced Elastic Weight Consolidation)
Prevents catastrophic forgetting:
```
Loss = Task_Loss + λ * Σᵢ Fᵢ(θᵢ - θ*ᵢ)²
Where:
• Fᵢ = Online Fisher information (EMA decay 0.999)
• θ*ᵢ = Optimal weights for previous tasks
• λ = Adaptive (2000 default, range 100-15000)
• Multi-task memory with circular buffer (10 tasks)
• Automatic task boundary detection
```
### SIMD Operations
Native CPU acceleration:
```rust
// AVX2 dot product (8 floats at a time)
#[target_feature(enable = "avx2")]
unsafe fn dot_product_avx2(a: &[f32], b: &[f32]) -> f32
// SSE4.1 fallback (4 floats at a time)
#[target_feature(enable = "sse4.1")]
unsafe fn dot_product_sse(a: &[f32], b: &[f32]) -> f32
// Automatic detection and dispatch
let result = SimdOps::dot_product(&a, &b);
```
---
## Supported Models
### Real Inference (CPU SIMD)
| SmolLM 135M | 135M | 2048 | HuggingFaceTB/SmolLM-135M |
| SmolLM 360M | 360M | 2048 | HuggingFaceTB/SmolLM-360M |
| Qwen2 0.5B | 500M | 4096 | Qwen/Qwen2-0.5B |
| TinyLlama 1.1B | 1.1B | 2048 | TinyLlama/TinyLlama-1.1B-Chat |
All models support Q4_K_M quantization for efficient CPU inference.
---
## HTTP Server API
When running with the `server` feature:
| `/health` | GET | Health check |
| `/query` | POST | Submit query |
| `/stats` | GET | Get statistics |
| `/feedback` | POST | Submit feedback |
| `/session` | POST | Create new session |
```bash
# Example query
curl -X POST http://localhost:3000/query \
-H "Content-Type: application/json" \
-d '{"query": "What is Rust?", "session_id": null}'
```
---
## Testing
```bash
# Run all tests
cargo test -p ruvllm
# Unit tests only (47 tests)
cargo test -p ruvllm --lib
# Integration tests (15 tests)
cargo test -p ruvllm --test integration
# With output
cargo test -p ruvllm -- --nocapture
```
### Test Coverage
| Memory (HNSW) | 12 | Search, insertion, graph expansion |
| Router (FastGRNN) | 8 | Forward pass, training, EWC |
| Attention | 6 | Multi-head, edge features, cross-attention |
| Embedding | 9 | Tokenization, caching, pooling |
| SONA | 10 | LoRA, EWC++, ReasoningBank, loops |
| Orchestrator | 2 | End-to-end pipeline |
| Integration | 15 | Full system tests |
---
## Project Structure
```
examples/ruvLLM/
├── Cargo.toml # Dependencies and features
├── README.md # This file
├── src/
│ ├── lib.rs # Library entry point
│ ├── config.rs # Configuration system
│ ├── error.rs # Error types
│ ├── types.rs # Core domain types
│ ├── orchestrator.rs # Main RuvLLM coordinator
│ ├── memory.rs # HNSW memory service
│ ├── router.rs # FastGRNN router
│ ├── attention.rs # Graph attention engine
│ ├── embedding.rs # Embedding service
│ ├── inference.rs # Mock inference pool
│ ├── inference_real.rs # Candle-based real inference
│ ├── simd_inference.rs # SIMD-optimized transformer
│ ├── learning.rs # Self-learning service
│ ├── compression.rs # Memory compression
│ ├── training.rs # Pretraining pipeline
│ ├── trm/ # TRM (Tiny Recursive Models) module
│ │ ├── mod.rs # Module exports and traits
│ │ ├── engine.rs # Main TRM reasoning engine
│ │ ├── config.rs # Configuration and builder
│ │ ├── mlp.rs # MLP latent updater
│ │ ├── attention.rs # Attention latent updater
│ │ ├── refiner.rs # Answer refinement
│ │ ├── confidence.rs # Confidence scoring
│ │ ├── sona_bridge.rs # SONA integration
│ │ ├── types.rs # TRM types and results
│ │ └── error.rs # Error handling
│ ├── sona/ # SONA module
│ │ ├── mod.rs # Module exports
│ │ ├── types.rs # SONA types
│ │ ├── lora.rs # MicroLoRA & BaseLoRA
│ │ ├── ewc.rs # EWC++ implementation
│ │ ├── reasoning_bank.rs # Pattern storage
│ │ ├── trajectory.rs # Trajectory recording
│ │ ├── engine.rs # SONA engine
│ │ └── loops/ # Temporal learning loops
│ │ ├── instant.rs # Per-request loop
│ │ ├── background.rs # Hourly loop
│ │ └── coordinator.rs # Loop coordinator
│ └── bin/
│ ├── demo.rs # Interactive demo
│ ├── bench.rs # Quick benchmarks
│ ├── benchmark_suite.rs # Full benchmark suite
│ ├── simd_demo.rs # SIMD capabilities demo
│ ├── pretrain.rs # Pretraining pipeline
│ ├── export.rs # HuggingFace export
│ └── server.rs # HTTP server
├── tests/
│ └── integration.rs # Integration tests
├── benches/
│ ├── pipeline.rs # Full pipeline benchmarks
│ ├── router.rs # Router benchmarks
│ ├── memory.rs # Memory benchmarks
│ ├── attention.rs # Attention benchmarks
│ ├── sona_bench.rs # SONA benchmarks
│ └── trm_bench.rs # TRM benchmarks
├── config/ # Configuration files
└── docs/
└── sparc/ # SPARC methodology docs
```
---
## Feature Flags
### RuvLLM Features
| `storage` | ✓ | Persistent storage and HNSW indexing |
| `metrics` | ✓ | Prometheus metrics export |
| `server` | ✗ | HTTP server with Axum |
| `real-inference` | ✗ | Candle-based real LLM inference |
| `hf-export` | ✗ | HuggingFace export via ruvector-sona |
| `full` | ✗ | All features enabled |
```bash
# Build with all features
cargo build --release --features full
```
### ruvector-sona Features (Dependency)
| `serde-support` | ✓ | Serialization for export, training, and federated learning |
| `wasm` | ✗ | WebAssembly bindings for browser/edge deployment |
| `napi` | ✗ | N-API bindings for Node.js integration |
```bash
# Build SONA with WASM support
cd crates/sona
wasm-pack build --target web --features wasm
```
---
## Configuration Options
| `embedding.dimension` | 768 | Embedding vector size |
| `embedding.max_tokens` | 512 | Max tokens per input |
| `memory.hnsw_m` | 16 | HNSW connections per node |
| `memory.hnsw_ef_construction` | 100 | Build quality parameter |
| `memory.hnsw_ef_search` | 64 | Search quality parameter |
| `router.input_dim` | 128 | Router input features |
| `router.hidden_dim` | 64 | FastGRNN hidden size |
| `router.sparsity` | 0.9 | Weight matrix sparsity |
| `router.rank` | 8 | Low-rank decomposition |
| `learning.enabled` | true | Enable self-learning |
| `learning.quality_threshold` | 0.7 | Min quality for writeback |
| `learning.ewc_lambda` | 2000 | EWC regularization strength |
| `sona.pattern_clusters` | 100 | K-means++ clusters |
| `sona.micro_lora_rank` | 2 | MicroLoRA rank |
### Federated Learning Configuration
| `federated.quality_threshold` | 0.4 | Min quality for trajectory acceptance |
| `federated.consolidation_interval` | 50 | Auto-consolidate every N agents |
| `federated.coordinator_capacity` | 50000 | Trajectory buffer size for coordinator |
| `federated.agent_capacity` | 500 | Trajectory buffer size per agent |
| `federated.base_lora_rank` | 16 | Coordinator LoRA rank (deeper for aggregation) |
---
## Self-Learning Improvement Over Time
| 0 | 0 | 65.0% | 50.0% | 0.0% | 0 | 0.0% (baseline) |
| 1 | 50 | 67.2% | 58.0% | 10.0% | 25 | +3.4% |
| 2 | 100 | 69.8% | 66.0% | 20.0% | 50 | +7.4% |
| 3 | 150 | 71.5% | 74.0% | 30.0% | 75 | +10.0% |
| 4 | 200 | 73.2% | 82.0% | 40.0% | 100 | +12.6% |
| 5 | 250 | 74.8% | 90.0% | 50.0% | 125 | +15.1% |
---
## References
- [TinyRecursiveModels](https://github.com/SamsungSAILMontreal/TinyRecursiveModels) - Samsung SAIL Montreal's recursive reasoning approach
- [LFM2: Liquid Foundation Models](https://arxiv.org/abs/2511.23404v1) - Gated convolutions + grouped query attention
- [FastGRNN](https://arxiv.org/abs/1901.02358) - Fast, Accurate, Stable and Tiny GRU
- [HNSW](https://arxiv.org/abs/1603.09320) - Hierarchical Navigable Small World Graphs
- [EWC](https://arxiv.org/abs/1612.00796) - Elastic Weight Consolidation
- [LoRA](https://arxiv.org/abs/2106.09685) - Low-Rank Adaptation of Large Language Models
---
## License
Licensed under either of:
- Apache License, Version 2.0 ([LICENSE-APACHE](LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0)
- MIT license ([LICENSE-MIT](LICENSE-MIT) or http://opensource.org/licenses/MIT)
at your option.
## Contributing
Contributions are welcome! Please feel free to submit a Pull Request.
---
<p align="center">
<b>Built with Rust + Ruvector</b><br>
<i>Self-Learning AI that gets smarter with every interaction</i>
</p>