Skip to main content

LlmProvider

embacle::types

Trait LlmProvider 

Source 
pub trait LlmProvider: Send + Sync {
    // Required methods
    fn name(&self) -> &'static str;
    fn display_name(&self) -> &str;
    fn capabilities(&self) -> LlmCapabilities;
    fn default_model(&self) -> &str;
    fn available_models(&self) -> &[String];
    fn complete<'life0, 'life1, 'async_trait>(
        &'life0 self,
        request: &'life1 ChatRequest,
    ) -> Pin<Box<dyn Future<Output = Result<ChatResponse, RunnerError>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait,
             'life1: 'async_trait;
    fn complete_stream<'life0, 'life1, 'async_trait>(
        &'life0 self,
        request: &'life1 ChatRequest,
    ) -> Pin<Box<dyn Future<Output = Result<ChatStream, RunnerError>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait,
             'life1: 'async_trait;
    fn health_check<'life0, 'async_trait>(
        &'life0 self,
    ) -> Pin<Box<dyn Future<Output = Result<bool, RunnerError>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait;

    // Provided method
    fn complete_stream_with_tools<'life0, 'life1, 'async_trait>(
        &'life0 self,
        request: &'life1 ChatRequest,
    ) -> Pin<Box<dyn Future<Output = Result<ToolStream, RunnerError>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait,
             'life1: 'async_trait { ... }
}
Expand description

LLM provider trait for chat completion

Implement this trait to add a new LLM runner. Each runner wraps a CLI tool and translates between the chat protocol and the tool’s native interface.

Required Methods§

Source

fn name(&self) -> &'static str

Unique provider identifier (e.g., claude_code, copilot)

Source

fn display_name(&self) -> &str

Human-readable display name for the provider

Source

fn capabilities(&self) -> LlmCapabilities

Provider capabilities (streaming, function calling, etc.)

Source

fn default_model(&self) -> &str

Default model to use if not specified in request

Source

fn available_models(&self) -> &[String]

Available models for this provider

Source

fn complete<'life0, 'life1, 'async_trait>( &'life0 self, request: &'life1 ChatRequest, ) -> Pin<Box<dyn Future<Output = Result<ChatResponse, RunnerError>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Perform a chat completion (non-streaming)

Source

fn complete_stream<'life0, 'life1, 'async_trait>( &'life0 self, request: &'life1 ChatRequest, ) -> Pin<Box<dyn Future<Output = Result<ChatStream, RunnerError>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Perform a streaming chat completion

Returns a stream of chunks that can be consumed incrementally. Falls back to non-streaming if not supported.

Source

fn health_check<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Result<bool, RunnerError>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait,

Check if the provider is healthy and ready to serve requests

Provided Methods§

Source

fn complete_stream_with_tools<'life0, 'life1, 'async_trait>( &'life0 self, request: &'life1 ChatRequest, ) -> Pin<Box<dyn Future<Output = Result<ToolStream, RunnerError>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Perform a chat completion that may call tools, streaming its text.

Yields the reply text as ToolStreamEvent::TextDeltas while it is generated, then one ToolStreamEvent::Done carrying the response complete would have returned for request — tool calls (request.tools are offered as for complete), finish reason and token usage included.

The default implementation calls complete and yields its response as the single Done, with no deltas, so every provider answers this method; a provider that can stream tool use natively overrides it. A decorator overrides it to forward to the provider it wraps.

§Errors

An Err here means no stream was opened (the request was refused, the connection failed); a fallback chain moves to its next tier on it as it does for complete. A failure once the stream is open is an Err item, and the text already delivered cannot be taken back.

Dyn Compatibility§

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementations on Foreign Types§

Source§

impl<T: LlmProvider + ?Sized> LlmProvider for Arc<T>

A shared provider is a provider.

Lets an Arc<T> be boxed as a dyn LlmProvider tier — a fallback chain can hold the same runner a caller keeps a handle to (for its ACP session, its router state) without a wrapper type whose only job is to forward nine methods.

Source§

fn name(&self) -> &'static str

Source§

fn display_name(&self) -> &str

Source§

fn capabilities(&self) -> LlmCapabilities

Source§

fn default_model(&self) -> &str

Source§

fn available_models(&self) -> &[String]

Source§

fn complete<'life0, 'life1, 'async_trait>( &'life0 self, request: &'life1 ChatRequest, ) -> Pin<Box<dyn Future<Output = Result<ChatResponse, RunnerError>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Source§

fn complete_stream<'life0, 'life1, 'async_trait>( &'life0 self, request: &'life1 ChatRequest, ) -> Pin<Box<dyn Future<Output = Result<ChatStream, RunnerError>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Source§

fn complete_stream_with_tools<'life0, 'life1, 'async_trait>( &'life0 self, request: &'life1 ChatRequest, ) -> Pin<Box<dyn Future<Output = Result<ToolStream, RunnerError>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Source§

fn health_check<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Result<bool, RunnerError>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait,

Implementors§