Struct ModelRouter 

pub struct ModelRouter { /* private fields */ }

Smart request router: selects the best model endpoint for a request.

Wraps a ModelRegistry and adds routing logic including fallback to the default model, context-length awareness, and OpenAI-compatible model listing.

Implementations

impl ModelRouter

pub fn new(registry: ModelRegistry) -> Self

Create a new router backed by the given registry.

pub fn route( &self, requested_model: Option<&str>, ) -> Result<&ModelEndpoint, RoutingError>

Route a request: resolve model_id from the request.

Falls back to the default model if requested_model is None. Returns an error if the resolved endpoint is not in a Ready state.

pub fn route_for_context( &self, requested_model: Option<&str>, required_context: usize, ) -> Result<&ModelEndpoint, RoutingError>

Route with context-length awareness: pick a model that can accommodate the required context length.

If a specific model is requested, validates it has sufficient context. If no model is specified, finds the default model that fits, or falls back to any available model with sufficient context capacity.

pub fn models_list(&self) -> Vec<ModelListEntry>

OpenAI-compatible /v1/models list.

Returns an entry for every available endpoint in the registry.

pub fn registry(&self) -> &ModelRegistry

Immutable access to the underlying registry.

pub fn registry_mut(&mut self) -> &mut ModelRegistry

Mutable access to the underlying registry.