Struct ProviderRouter 

pub struct ProviderRouter { /* private fields */ }

Routes LLM requests across multiple providers.

Implementations

impl ProviderRouter

pub fn new(strategy: RoutingStrategy) -> Self

Create a new router with the given strategy.

pub fn with_provider(self, entry: ProviderEntry) -> Self

Add a provider to the router.

pub fn provider_count(&self) -> usize

Returns the number of registered providers.

Trait Implementations

impl Debug for ProviderRouter

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl LlmPort for ProviderRouter

fn capabilities(&self) -> LlmCapabilities

Runtime capability declaration for dispatch decisions.

fn complete<'life0, 'async_trait>( &'life0 self, req: LlmRequest, ) -> Pin<Box<dyn Future<Output = Result<LlmResponse, LlmError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Run a non-streaming inference call.

fn complete_stream<'life0, 'async_trait>( &'life0 self, req: LlmRequest, ) -> Pin<Box<dyn Future<Output = Result<LlmStream, LlmError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Run a streaming inference call.