Struct RuvLLMEngine 

pub struct RuvLLMEngine { /* private fields */ }

Main RuvLLM engine for LLM inference with intelligent memory.

The RuvLLMEngine is the primary entry point for RuvLLM, providing:

• Session Management: Create and manage user sessions with state persistence
• Policy Storage: Ruvector-backed semantic search for runtime policies
• Adapter Management: Hot-swapping LoRA adapters for task-specific tuning
• Witness Logging: Audit trail with HNSW-indexed semantic search
• SONA Learning: Three-tier continuous learning integration

Example

use ruvllm::{RuvLLMEngine, RuvLLMConfig};

// Create engine with configuration
let config = RuvLLMConfig::default();
let engine = RuvLLMEngine::new(config)?;

// Create a session for a user
let session = engine.create_session(Some("user-123"))?;

// Search for relevant policies
let embedding = compute_embedding("code completion task");
let policies = engine.search_policies(&embedding, 5)?;

// Record audit entry
let entry = WitnessEntry::new("completion", latency, routing);
engine.record_witness(entry)?;

Architecture

+-------------------+     +-------------------+
| RuvLLMEngine      |---->| PolicyStore       |
|                   |     | (Ruvector)        |
|                   |     +-------------------+
|                   |
|                   |---->| SessionIndex      |
|                   |     | (Ruvector)        |
|                   |     +-------------------+
|                   |
|                   |---->| WitnessLog        |
|                   |     | (HNSW search)     |
+-------------------+     +-------------------+

Implementations

impl RuvLLMEngine

pub fn new(config: RuvLLMConfig) -> Result<Self>

Create a new RuvLLM engine with the given configuration.

This initializes all subsystems including:

• Policy store for learned thresholds
• Session index for conversation state
• Witness log for audit trails
• SONA integration for learning loops

Arguments

• config - Engine configuration

Errors

Returns an error if storage paths cannot be created or initialized.

Example

use ruvllm::{RuvLLMEngine, RuvLLMConfig};

let engine = RuvLLMEngine::new(RuvLLMConfig::default())?;

pub fn create_session(&self, user_id: Option<&str>) -> Result<Session>

Create a new session for a user.

Sessions track conversation state, KV cache references, and enable multi-turn interactions. Each session is automatically indexed in Ruvector for semantic retrieval.

Arguments

• user_id - Optional user identifier for session tracking

Returns

A new Session instance with a unique ID.

Example

// Anonymous session
let session = engine.create_session(None)?;

// User-identified session
let session = engine.create_session(Some("user-123"))?;
println!("Session ID: {}", session.id());

pub fn get_session(&self, session_id: &str) -> Result<Option<Session>>

Get session by ID

pub fn search_policies( &self, context_embedding: &[f32], limit: usize, ) -> Result<Vec<PolicyEntry>>

Search for policies matching the given context embedding.

Uses HNSW-indexed semantic search to find relevant policies (quantization settings, routing rules, etc.) based on the current request context.

Arguments

• context_embedding - Vector embedding of the current context
• limit - Maximum number of policies to return

Returns

Vector of matching PolicyEntry items, sorted by relevance.

Example

let context = compute_embedding("code completion for Python");
let policies = engine.search_policies(&context, 5)?;

for policy in policies {
    println!("Policy: {:?}, score: {}", policy.policy_type, policy.score);
}

pub fn record_witness(&self, entry: WitnessEntry) -> Result<()>

Record a witness entry for audit logging.

Witness entries provide an audit trail of inference decisions, including latency breakdowns, routing decisions, and quality scores. All entries are HNSW-indexed for semantic search.

Arguments

• entry - The witness entry to record

Example

use ruvllm::{WitnessEntry, LatencyBreakdown, RoutingDecision};

let entry = WitnessEntry {
    session_id: session.id().to_string(),
    request_type: "completion".to_string(),
    latency: LatencyBreakdown {
        prefill_ms: 45.0,
        decode_ms: 120.0,
        total_ms: 165.0,
    },
    routing: RoutingDecision::default(),
    ..Default::default()
};

engine.record_witness(entry)?;

pub fn search_witness( &self, query_embedding: &[f32], limit: usize, ) -> Result<Vec<WitnessEntry>>

Search witness logs semantically

pub fn sona(&self) -> &SonaIntegration

Get the SONA integration for learning

pub fn adapters(&self) -> &AdapterManager

Get the adapter manager

pub fn policies(&self) -> &PolicyStore

Get the policy store