Skip to main content

ClientRouter

Struct ClientRouter 

Source
pub struct ClientRouter { /* private fields */ }
Expand description

Resolves a routed call’s selected model to the client that serves it.

An algorithm routes among named targets; which provider each target lives on is the host’s concern, and two targets in one run may sit on different providers. A router owns that mapping. It is not itself a client: it hands back a RoutedLlmClient and the caller makes the call.

Cloning is cheap — the mapping is shared, so one router can serve every request.

Implementations§

Source§

impl ClientRouter

Source

pub fn new(by_model: HashMap<ModelId, Arc<dyn RoutedLlmClient>>) -> Self

Build a router over model name -> client, for targets spread across providers.

Source

pub fn new_with_target_prompts( by_model: HashMap<ModelId, Arc<dyn RoutedLlmClient>>, target_prompts: HashMap<ModelId, String>, routing_answer_target: Option<ModelId>, ) -> Self

Build a router with target prompts used by run and decide.

routing_answer_target identifies a routing-time call whose response may become the answer. Pass None when routing only selects a later completion target. Prompt keys and routing_answer_target use the resolved model IDs stored in by_model. This constructor treats every model in by_model as a possible completion target. Use Self::new_with_completion_targets to exclude classifier-only models from the comparison of completion clients.

Source

pub fn new_with_completion_targets( by_model: HashMap<ModelId, Arc<dyn RoutedLlmClient>>, target_prompts: HashMap<ModelId, String>, routing_answer_target: Option<ModelId>, completion_targets: &[ModelId], ) -> Self

Build a router with an explicit list of models that can answer requests.

by_model contains every callable model, including classifiers. Only the model IDs in completion_targets determine whether native Responses state needs a routing pin. Cross-format state is recorded lazily when a Responses request uses a Chat or Anthropic target. target_prompts and routing_answer_target have the same meaning as in Self::new_with_target_prompts.

Source

pub fn single(client: Arc<dyn RoutedLlmClient>) -> Self

A router that serves every model with one client — the single-provider case.

TranslatingLlmClient already maps model names to backends internally and rejects ones it does not know, so enumerating them here would only duplicate that.

Source

pub fn route( &self, model: &ModelId, ) -> Result<&Arc<dyn RoutedLlmClient>, LlmClientError>

The client that serves model.

Errors with LlmClientError::Configuration when the router maps models and has no entry for this one, rather than silently sending the call to another provider.

Trait Implementations§

Source§

impl Clone for ClientRouter

Source§

fn clone(&self) -> Self

Returns a duplicate of the value. Read more
1.0.0 (const: unstable) · Source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
Source§

impl FromIterator<(ModelId, Arc<dyn RoutedLlmClient>)> for ClientRouter

Source§

fn from_iter<I: IntoIterator<Item = (ModelId, Arc<dyn RoutedLlmClient>)>>( iter: I, ) -> Self

Creates a value from an iterator. Read more

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
where ST: ?Sized, DT: ?Sized,

Source§

impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
where ST: ?Sized, DT: ?Sized,

Source§

impl<T> CloneToUninit for T
where T: Clone,

Source§

unsafe fn clone_to_uninit(&self, dest: *mut u8)

🔬This is a nightly-only experimental API. (clone_to_uninit)
Performs copy-assignment from self to dest. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T> FutureExt for T

Source§

fn with_context(self, otel_cx: Context) -> WithContext<Self> ⓘ

Attaches the provided Context to this type, returning a WithContext wrapper. Read more
Source§

fn with_current_context(self) -> WithContext<Self> ⓘ

Attaches the current Context to this type, returning a WithContext wrapper. Read more
Source§

impl<T> Instrument for T

Source§

fn instrument(self, span: Span) -> Instrumented<Self> ⓘ

Instruments this type with the provided Span, returning an Instrumented wrapper. Read more
Source§

fn in_current_span(self) -> Instrumented<Self> ⓘ

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> PolicyExt for T
where T: ?Sized,

Source§

fn and<P, B, E>(self, other: P) -> And<T, P>
where T: Sized + Policy<B, E>, P: Policy<B, E>,

Create a new Policy that returns Action::Follow only if self and other return Action::Follow. Read more
Source§

fn or<P, B, E>(self, other: P) -> Or<T, P>
where T: Sized + Policy<B, E>, P: Policy<B, E>,

Create a new Policy that returns Action::Follow if either self or other returns Action::Follow. Read more
Source§

impl<T> Read<Exclusive, BecauseExclusive> for T
where T: ?Sized,

Source§

impl<T> ToOwned for T
where T: Clone,

Source§

type Owned = T

The resulting type after obtaining ownership.
Source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
Source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = !

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, !>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
Source§

impl<T> WithSubscriber for T

Source§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self> ⓘ
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a WithDispatch wrapper. Read more
Source§

fn with_current_subscriber(self) -> WithDispatch<Self> ⓘ

Attaches the current default Subscriber to this type, returning a WithDispatch wrapper. Read more