Struct ModelRouterMiddleware 

pub struct ModelRouterMiddleware { /* private fields */ }

Channel selection and model routing middleware.

Implementations

impl ModelRouterMiddleware

pub async fn from_storage(storage: Arc<dyn Storage>) -> Result<Self, ProxyError>

Creates a new ModelRouterMiddleware from a storage backend.

Loads all enabled channels and their model mappings, parsing the storage-layer string fields into typed enums.

Errors

Returns ProxyError if the storage backend fails or if any channel has an unrecognized protocol string.

pub fn health_map(&self) -> &Arc<DashMap<String, ChannelState>>

Returns a reference to the in-memory health map.

pub fn api_key_map(&self) -> &Arc<DashMap<String, SecretString>>

Returns a reference to the shared API-key override map.

The admin API writes updated keys here so the router picks them up at request time without needing a restart.

pub fn channels_swap(&self) -> Arc<ArcSwap<Vec<ResolvedChannel>>>

Returns a clone of the atomic channel list Arc so the admin API can trigger a hot-reload after mutations (priority, enabled, etc.).

Trait Implementations

impl Debug for ModelRouterMiddleware

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl ProxyMiddleware for ModelRouterMiddleware

fn on_request<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, req: &'life1 mut ProxyRequest, ctx: &'life2 mut ConnectionContext, ) -> Pin<Box<dyn Future<Output = Result<(), ProxyError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait,

Called before forwarding the request to upstream.

fn on_response<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, res: &'life1 mut ProxyResponse, ctx: &'life2 ConnectionContext, ) -> Pin<Box<dyn Future<Output = Result<(), ProxyError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait,

Called after receiving the response from upstream.

fn name(&self) -> &'static str

Returns the unique name of this middleware.

fn on_connect<'life0, 'life1, 'async_trait>( &'life0 self, _ctx: &'life1 ConnectionContext, ) -> Pin<Box<dyn Future<Output = ()> + Send + 'async_trait>>

where 'life0: 'async_trait, 'life1: 'async_trait, Self: 'async_trait,

Called when a new connection is established. Runs in registration order.

fn on_disconnect<'life0, 'life1, 'async_trait>( &'life0 self, _ctx: &'life1 ConnectionContext, ) -> Pin<Box<dyn Future<Output = ()> + Send + 'async_trait>>

where 'life0: 'async_trait, 'life1: 'async_trait, Self: 'async_trait,

Called when a connection is closed. Runs in reverse registration order.

fn on_init<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Result<(), ProxyError>> + Send + 'async_trait>>

where 'life0: 'async_trait, Self: 'async_trait,

Called once when the proxy starts. Use for opening DB pools, loading config, etc.

fn on_shutdown<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Result<(), ProxyError>> + Send + 'async_trait>>

where 'life0: 'async_trait, Self: 'async_trait,

Called once when the proxy shuts down gracefully.