LLMProvider

umi_memory::llm

Trait LLMProvider 

Source 
pub trait LLMProvider: Send + Sync {
    // Required methods
    fn complete<'life0, 'life1, 'async_trait>(
        &'life0 self,
        request: &'life1 CompletionRequest,
    ) -> Pin<Box<dyn Future<Output = Result<String, ProviderError>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait,
             'life1: 'async_trait;
    fn name(&self) -> &'static str;
    fn is_simulation(&self) -> bool;

    // Provided method
    fn complete_json<'life0, 'life1, 'async_trait, T>(
        &'life0 self,
        request: &'life1 CompletionRequest,
    ) -> Pin<Box<dyn Future<Output = Result<T, ProviderError>> + Send + 'async_trait>>
       where T: 'async_trait + DeserializeOwned + Send,
             Self: 'async_trait,
             'life0: 'async_trait,
             'life1: 'async_trait { ... }
}
Expand description

Trait for LLM providers.

TigerStyle: Unified interface for simulation and production.

All providers implement this trait, allowing higher-level components to work with any provider without knowing the concrete type.

§Example

use umi_memory::llm::{LLMProvider, SimLLMProvider, CompletionRequest};

async fn extract_entities<P: LLMProvider>(provider: &P, text: &str) -> String {
    let request = CompletionRequest::new(format!("Extract entities from: {}", text));
    provider.complete(&request).await.unwrap()
}

Required Methods§

Source

fn complete<'life0, 'life1, 'async_trait>( &'life0 self, request: &'life1 CompletionRequest, ) -> Pin<Box<dyn Future<Output = Result<String, ProviderError>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Complete a prompt with a text response.

§Errors

Returns ProviderError on failure.

Source

fn name(&self) -> &'static str

Get the provider name for logging/debugging.

Source

fn is_simulation(&self) -> bool

Check if this is a simulation provider.

Returns true for SimLLMProvider, false for real providers.

Provided Methods§

Source

fn complete_json<'life0, 'life1, 'async_trait, T>( &'life0 self, request: &'life1 CompletionRequest, ) -> Pin<Box<dyn Future<Output = Result<T, ProviderError>> + Send + 'async_trait>>
where T: 'async_trait + DeserializeOwned + Send, Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Complete a prompt expecting a JSON response.

This is a convenience method that calls complete and parses the response.

§Errors

Returns ProviderError on failure or JSON parse error.

Dyn Compatibility§

This trait is not dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety", so this trait is not object safe.

Implementors§