Trait ModelProvider 

pub trait ModelProvider: Send + Sync {
    // Required method
    fn invoke<'life0, 'async_trait>(
        &'life0 self,
        request: ModelRequest,
    ) -> Pin<Box<dyn Future<Output = Result<ModelResponse>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait;

    // Provided method
    fn stream(&self, request: ModelRequest) -> StreamEventStream { ... }
}

The provider abstraction. Implement this to add a backend.

Flue routes every model interaction through pi-ai’s Model interface; fluers-core does the same through this trait.

Required Methods

fn invoke<'life0, 'async_trait>( &'life0 self, request: ModelRequest, ) -> Pin<Box<dyn Future<Output = Result<ModelResponse>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Run a turn, returning the full response.

Provided Methods

fn stream(&self, request: ModelRequest) -> StreamEventStream

Stream a turn as events. The default buffers [invoke]; providers with native streaming override this to emit deltas as they arrive.

Dyn Compatibility

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors