Trait InferenceProvider 

pub trait InferenceProvider: Send + Sync {
    // Required methods
    fn infer<'life0, 'life1, 'async_trait>(
        &'life0 self,
        req: &'life1 InferenceRequest,
    ) -> Pin<Box<dyn Future<Output = Result<InferenceResponse, Box<dyn Error + Send + Sync>>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait,
             'life1: 'async_trait;
    fn name(&self) -> &str;
    fn model(&self) -> &ModelId;
    fn validate_config(&self) -> Result<(), Box<dyn Error + Send + Sync>>;

    // Provided method
    fn supports_tool_use(&self) -> bool { ... }
}

Port: perform LLM inference.

Implementations decide the backend (Anthropic, OpenAI, local Ollama, etc.).

Required Methods

fn infer<'life0, 'life1, 'async_trait>( &'life0 self, req: &'life1 InferenceRequest, ) -> Pin<Box<dyn Future<Output = Result<InferenceResponse, Box<dyn Error + Send + Sync>>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Run inference with the given request.

fn name(&self) -> &str

Get the name of this provider.

fn model(&self) -> &ModelId

Get the model being used.

fn validate_config(&self) -> Result<(), Box<dyn Error + Send + Sync>>

Validate that this provider is properly configured.

Provided Methods

fn supports_tool_use(&self) -> bool

Whether this provider supports tool use (function calling).

Dyn Compatibility

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors