Skip to main content

HasInference

roboticus_api::api::state

Trait HasInference 

Source 
pub trait HasInference {
    // Required methods
    fn llm(&self) -> &Arc<RwLock<LlmService>>;
    fn semantic_classifier(&self) -> &Arc<SemanticClassifier>;
}
Expand description

LLM inference: model routing, quality tracking, latency, caching.

Required Methods§

Source

fn llm(&self) -> &Arc<RwLock<LlmService>>

Source

fn semantic_classifier(&self) -> &Arc<SemanticClassifier>

Implementors§