Struct InferenceEngine 

pub struct InferenceEngine {
    pub config: InferenceConfig,
    pub unified_registry: UnifiedRegistry,
    pub adaptive_router: AdaptiveRouter,
    pub outcome_tracker: Arc<RwLock<OutcomeTracker>>,
    pub registry: ModelRegistry,
    pub router: ModelRouter,
    /* private fields */
}

The main inference engine. Thread-safe, lazily loads models.

Now includes the unified registry, adaptive router, and outcome tracker for schema-driven model selection with learned performance profiles.

Fields

config: InferenceConfig

unified_registry: UnifiedRegistry

Unified model registry (local + remote).

adaptive_router: AdaptiveRouter

Adaptive router with three-phase selection.

outcome_tracker: Arc<RwLock<OutcomeTracker>>

Outcome tracker for learning from results.

registry: ModelRegistry

router: ModelRouter

Implementations

impl InferenceEngine

pub fn new(config: InferenceConfig) -> Self

pub async fn route_adaptive(&self, prompt: &str) -> AdaptiveRoutingDecision

Route a prompt using the adaptive router (new). Returns full decision context.

pub fn route(&self, prompt: &str) -> RoutingDecision

Route a prompt to the best model without executing (legacy compat).

pub async fn generate_tracked( &self, req: GenerateRequest, ) -> Result<InferenceResult, InferenceError>

Generate text from a prompt with outcome tracking. Returns InferenceResult with trace_id for reporting outcomes.

pub async fn generate( &self, req: GenerateRequest, ) -> Result<String, InferenceError>

Generate text from a prompt (legacy API, no outcome tracking). When req.model is None, uses intelligent routing based on prompt complexity.

pub async fn embed( &self, req: EmbedRequest, ) -> Result<Vec<Vec<f32>>, InferenceError>

Generate embeddings for text using the dedicated embedding model. Uses Qwen3-Embedding with proper last-token hidden state extraction.

pub async fn classify( &self, req: ClassifyRequest, ) -> Result<Vec<ClassifyResult>, InferenceError>

Classify text against candidate labels. When req.model is None, routes to the smallest available model.

pub fn list_models_unified(&self) -> Vec<ModelInfo>

List all known models and their status (new registry).

pub fn list_models(&self) -> Vec<ModelInfo>

List all known models and their download status (legacy).

pub async fn pull_model(&self, name: &str) -> Result<PathBuf, InferenceError>

Download a model if not already present.

pub fn remove_model(&self, name: &str) -> Result<(), InferenceError>

Remove a downloaded model.

pub fn register_model(&mut self, schema: ModelSchema)

Register a model in the unified registry.

pub fn outcome_tracker(&self) -> Arc<RwLock<OutcomeTracker>>

Get outcome tracker for external use (e.g., memgine integration).

pub async fn export_profiles(&self) -> Vec<ModelProfile>

Export model performance profiles for persistence.

pub async fn import_profiles(&self, profiles: Vec<ModelProfile>)

Import model performance profiles (from persistence).