pub struct ClientRouter { /* private fields */ }Expand description
Resolves a routed call’s selected model to the client that serves it.
An algorithm routes among named targets; which provider each target lives on is the
host’s concern, and two targets in one run may sit on different providers. A router owns
that mapping. It is not itself a client: it hands back a RoutedLlmClient and the
caller makes the call.
Cloning is cheap — the mapping is shared, so one router can serve every request.
Implementations§
Source§impl ClientRouter
impl ClientRouter
Sourcepub fn new(by_model: HashMap<ModelId, Arc<dyn RoutedLlmClient>>) -> Self
pub fn new(by_model: HashMap<ModelId, Arc<dyn RoutedLlmClient>>) -> Self
Build a router over model name -> client, for targets spread across providers.
Sourcepub fn new_with_target_prompts(
by_model: HashMap<ModelId, Arc<dyn RoutedLlmClient>>,
target_prompts: HashMap<ModelId, String>,
routing_answer_target: Option<ModelId>,
) -> Self
pub fn new_with_target_prompts( by_model: HashMap<ModelId, Arc<dyn RoutedLlmClient>>, target_prompts: HashMap<ModelId, String>, routing_answer_target: Option<ModelId>, ) -> Self
Build a router with target prompts used by run and decide.
routing_answer_target identifies a routing-time call whose response may become the
answer. Pass None when routing only selects a later completion target. Prompt keys and
routing_answer_target use the resolved model IDs stored in by_model.
This constructor treats every model in by_model as a possible completion target.
Use Self::new_with_completion_targets to exclude classifier-only models from the
comparison of completion clients.
Sourcepub fn new_with_completion_targets(
by_model: HashMap<ModelId, Arc<dyn RoutedLlmClient>>,
target_prompts: HashMap<ModelId, String>,
routing_answer_target: Option<ModelId>,
completion_targets: &[ModelId],
) -> Self
pub fn new_with_completion_targets( by_model: HashMap<ModelId, Arc<dyn RoutedLlmClient>>, target_prompts: HashMap<ModelId, String>, routing_answer_target: Option<ModelId>, completion_targets: &[ModelId], ) -> Self
Build a router with an explicit list of models that can answer requests.
by_model contains every callable model, including classifiers. Only the model IDs in
completion_targets determine whether native Responses state needs a routing pin.
Cross-format state is recorded lazily when a Responses request uses a Chat or Anthropic
target. target_prompts and routing_answer_target have the same meaning as in
Self::new_with_target_prompts.
Sourcepub fn single(client: Arc<dyn RoutedLlmClient>) -> Self
pub fn single(client: Arc<dyn RoutedLlmClient>) -> Self
A router that serves every model with one client — the single-provider case.
TranslatingLlmClient already maps model names to
backends internally and rejects ones it does not know, so enumerating them here would
only duplicate that.
Sourcepub fn route(
&self,
model: &ModelId,
) -> Result<&Arc<dyn RoutedLlmClient>, LlmClientError>
pub fn route( &self, model: &ModelId, ) -> Result<&Arc<dyn RoutedLlmClient>, LlmClientError>
The client that serves model.
Errors with LlmClientError::Configuration when the router maps models and has no
entry for this one, rather than silently sending the call to another provider.