Struct ModelRouter 

pub struct ModelRouter { /* private fields */ }

Routes model requests (chat and embedding) across providers with capability checking, exponential-backoff retry, and fallback chains.

Implementations

impl ModelRouter

pub fn new(registry: Arc<ProviderRegistry>, policy: RuntimePolicy) -> Self

Creates a new model router.

pub fn registry(&self) -> &ProviderRegistry

Returns the provider registry.

pub fn policy(&self) -> &RuntimePolicy

Returns the runtime policy.

pub async fn route_chat( &self, provider_id: &ProviderId, request: ChatRequest, required_capabilities: Option<&ProviderCapabilities>, ) -> RuntimeResult<ChatResponse>

Routes a chat request to a provider with capability checking and retry.

Errors

Returns RuntimeError if provider not found, lacks capabilities, or all retries fail.

pub async fn route_chat_with_fallback( &self, provider_ids: &[ProviderId], request: ChatRequest, required_capabilities: Option<&ProviderCapabilities>, ) -> RuntimeResult<ChatResponse>

Routes a chat request across multiple providers with fallback ordering.

Each provider is tried in order via Self::route_chat (which itself applies per-provider retry logic). The first successful response is returned. When all providers fail, the last error is propagated.

Errors

Returns RuntimeError if every provider in the chain fails.

pub async fn route_embedding( &self, provider_id: &ProviderId, request: EmbeddingRequest, ) -> RuntimeResult<EmbeddingResponse>

Routes an embedding request with retry logic.

Errors

Returns RuntimeError if provider not found or all retries fail.