Struct RouterProvider 

pub struct RouterProvider { /* private fields */ }

Implementations

impl RouterProvider

pub fn new(providers: Vec<AnyProvider>) -> Self

pub fn with_embed_concurrency(self, limit: usize) -> Self

Set the maximum number of concurrent embed_batch calls.

A value of 0 disables the semaphore (unlimited). Default is no semaphore.

pub fn set_memory_confidence(&self, confidence: Option<f32>)

Set the MAR (Memory-Augmented Routing) signal for the current turn.

Must be called before chat / chat_stream to influence bandit provider selection. Pass None to disable MAR for this turn.

pub fn with_ema(self, alpha: f64, reorder_interval: u64) -> Self

Enable EMA-based adaptive provider ordering.

pub fn with_asi(self, config: AsiRouterConfig) -> Self

Enable Agent Stability Index (ASI) coherence tracking.

When enabled, each successful response is embedded in a background task and added to a per-provider sliding window. The coherence score (cosine similarity of the latest embedding vs. window mean) penalizes Thompson/EMA routing priors for providers whose responses drift.

pub fn with_quality_gate(self, threshold: f32) -> Self

Enable embedding-based quality gate for Thompson/EMA routing.

After provider selection, computes cosine similarity between the query embedding and the response embedding. If below threshold, tries the next provider in the ordered list. On full exhaustion, returns the best response seen (highest similarity). Fail-open: embedding errors disable the gate for that request.

pub fn with_thompson(self, state_path: Option<&Path>) -> Self

Enable Thompson Sampling strategy.

Loads existing state from state_path if present; falls back to uniform prior. Prunes stale entries for providers not in the current chain.

pub fn with_bandit( self, config: BanditRouterConfig, state_path: Option<&Path>, embedding_provider: Option<AnyProvider>, ) -> Self

Enable PILOT bandit routing strategy (LinUCB contextual bandit).

Loads existing state from state_path (or the default path). Applies session-level decay if config.decay_factor < 1.0, and prunes arms for removed providers.

embedding_provider is used to obtain feature vectors for each query. When None, the bandit falls back to Thompson/uniform selection whenever an embedding cannot be obtained within config.embedding_timeout_ms.

The warmup_queries default of 0 in BanditRouterConfig is overridden here to 10 * num_providers to ensure sufficient initial exploration.

pub fn save_bandit_state(&self)

Persist current bandit state to disk. No-op if bandit strategy is not active.

pub fn bandit_stats(&self) -> Vec<(String, u64, f32)>

Return bandit diagnostic stats: (provider_name, pulls, mean_reward).

Returns an empty vec if bandit strategy is not active.

pub fn with_reputation( self, decay_factor: f64, weight: f64, min_observations: u64, state_path: Option<&Path>, ) -> Self

Enable Bayesian reputation scoring (RAPS).

Loads existing state from state_path (or the default path), applies session-level decay, and prunes stale provider entries.

No-op for Cascade routing (reputation is not used for cost-tier ordering).

pub fn record_quality_outcome(&self, _provider_name: &str, success: bool)

Record a quality outcome for the last active sub-provider (tool execution result).

Call only for semantic failures (invalid tool args, parse errors). Do NOT call for network errors, rate limits, or transient I/O failures. No-op when reputation scoring is disabled, strategy is Cascade, or no tool call has been made yet in this session.

The _provider_name parameter is ignored — quality is attributed to the sub-provider that served the most recent chat_with_tools call, tracked via last_active_provider.

pub fn save_reputation_state(&self)

Persist current reputation state to disk. No-op if reputation is disabled.

pub fn reputation_stats(&self) -> Vec<(String, f64, f64, f64, u64)>

Return reputation stats for all tracked providers: (name, alpha, beta, mean, observations).

pub fn with_cascade(self, config: CascadeRouterConfig) -> Self

Enable Cascade routing strategy.

Providers are tried in chain order (cheapest first). Each response is evaluated by the quality classifier; if it falls below quality_threshold, the next provider is tried. At most max_escalations quality-based escalations occur.

Network/API errors do not count against the escalation budget. The best response seen so far is returned if all escalations are exhausted.

When config.cost_tiers is set, providers are reordered once at construction time (no per-request cost). Providers absent from cost_tiers are appended after listed ones in original chain order. Unknown names in cost_tiers are silently ignored.

pub fn save_thompson_state(&self)

Persist current Thompson state to disk.

No-op if Thompson strategy is not active.

Note

This performs synchronous I/O. Called at agent shutdown from an async context; acceptable since it runs after all in-flight requests have completed.

pub fn thompson_stats(&self) -> Vec<(String, f64, f64)>

Return a snapshot of Thompson distribution parameters for all tracked providers.

Returns an empty vec if Thompson strategy is not active.

pub fn set_status_tx(&mut self, tx: StatusTx)

pub async fn list_models_remote(&self) -> Result<Vec<RemoteModelInfo>, LlmError>

Aggregate model lists from all sub-providers, deduplicating by id.

Individual sub-provider errors are logged as warnings and skipped.

Errors

Always succeeds (errors per-provider are swallowed).

Trait Implementations

impl Clone for RouterProvider

fn clone(&self) -> RouterProvider

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for RouterProvider

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl LlmProvider for RouterProvider

fn context_window(&self) -> Option<usize>

Report the model’s context window size in tokens.

fn chat( &self, messages: &[Message], ) -> impl Future<Output = Result<String, LlmError>> + Send

Send messages to the LLM and return the assistant response.

fn chat_stream( &self, messages: &[Message], ) -> impl Future<Output = Result<ChatStream, LlmError>> + Send

Send messages and return a stream of response chunks.

fn supports_streaming(&self) -> bool

Whether this provider supports native streaming.

fn embed( &self, text: &str, ) -> impl Future<Output = Result<Vec<f32>, LlmError>> + Send

Generate an embedding vector from text.

fn embed_batch( &self, texts: &[&str], ) -> impl Future<Output = Result<Vec<Vec<f32>>, LlmError>> + Send

Embed multiple texts in a single API call.

fn supports_embeddings(&self) -> bool

Whether this provider supports embedding generation.

fn name(&self) -> &str

Provider name for logging and identification.

fn supports_tool_use(&self) -> bool

Whether this provider supports native tool_use / function calling.

fn list_models(&self) -> Vec<String>

Return the list of model identifiers this provider can serve. Default: empty (provider does not advertise models).

fn chat_with_tools( &self, messages: &[Message], tools: &[ToolDefinition], ) -> impl Future<Output = Result<ChatResponse, LlmError>> + Send

Send messages with tool definitions, returning a structured response.

fn debug_request_json( &self, messages: &[Message], tools: &[ToolDefinition], stream: bool, ) -> Value

Return the request payload that will be sent to the provider, for debug dumps.

fn last_cache_usage(&self) -> Option<(u64, u64)>

Return the cache usage from the last API call, if available. Returns (cache_creation_tokens, cache_read_tokens).

fn model_identifier(&self) -> &str

Model identifier string (e.g. gpt-4o-mini, claude-sonnet-4-6). Used by cost-estimation heuristics. Returns "" when not applicable.

fn supports_vision(&self) -> bool

Whether this provider supports image input (vision).

fn last_usage(&self) -> Option<(u64, u64)>

Return token counts from the last API call, if available. Returns (input_tokens, output_tokens).

fn take_compaction_summary(&self) -> Option<String>

Return the compaction summary from the most recent API call, if a server-side compaction occurred (Claude compact-2026-01-12 beta). Clears the stored value.

fn record_quality_outcome(&self, _provider_name: &str, _success: bool)

Record a quality outcome from tool execution for reputation-based routing (RAPS).

fn supports_structured_output(&self) -> bool

Whether this provider supports native structured output.

async fn chat_typed<T>(&self, messages: &[Message]) -> Result<T, LlmError>

where T: DeserializeOwned + JsonSchema + 'static, Self: Sized,

Send messages and parse the response into a typed value T.