pub struct ModelRouter { /* private fields */ }Expand description
Smart model router that picks cheap / mid / expensive models based on message complexity.
Implementations§
Source§impl ModelRouter
impl ModelRouter
Sourcepub fn new(config: ModelRoutingConfig) -> Self
pub fn new(config: ModelRoutingConfig) -> Self
Create a new router from the routing configuration.
Sourcepub fn is_enabled(&self) -> bool
pub fn is_enabled(&self) -> bool
Returns true if model routing is enabled.
Sourcepub fn classify(message: &str) -> ModelTier
pub fn classify(message: &str) -> ModelTier
Classify a user message into a complexity tier using keyword heuristics.
The classification is intentionally simple: pattern matching on the lowercased message. No LLM call is made.
Sourcepub fn classify_with_context(message: &str, messages: &[Message]) -> ModelTier
pub fn classify_with_context(message: &str, messages: &[Message]) -> ModelTier
Classify a user message with context awareness: if the conversation contains images (from screenshots, Telegram photos, etc.), force the expensive tier so a vision-capable model handles them.
Sourcepub fn select_model(&self, tier: ModelTier) -> Option<&ModelConfig>
pub fn select_model(&self, tier: ModelTier) -> Option<&ModelConfig>
Select the model config for a given tier. Returns None if the tier
has no model configured (caller should fall back to the default model).
Sourcepub fn route_message(&self, message: &str) -> Option<(ModelTier, ModelConfig)>
pub fn route_message(&self, message: &str) -> Option<(ModelTier, ModelConfig)>
Classify a message and create a driver for the selected tier.
Returns Some((tier, driver)) if routing is enabled and a tier-specific
model is configured. Returns None if routing is disabled or the tier
model is not configured (the caller should use the default driver).
Sourcepub fn route_message_with_context(
&self,
message: &str,
messages: &[Message],
) -> Option<(ModelTier, ModelConfig)>
pub fn route_message_with_context( &self, message: &str, messages: &[Message], ) -> Option<(ModelTier, ModelConfig)>
Classify a message with conversation context and return the tier-specific model config. This enables image-aware routing where conversations containing screenshots or photos automatically escalate to vision-capable models.
Sourcepub fn create_tier_driver(
config: &ModelConfig,
) -> PunchResult<Arc<dyn LlmDriver>>
pub fn create_tier_driver( config: &ModelConfig, ) -> PunchResult<Arc<dyn LlmDriver>>
Create an LLM driver for a routed model config.
Auto Trait Implementations§
impl Freeze for ModelRouter
impl RefUnwindSafe for ModelRouter
impl Send for ModelRouter
impl Sync for ModelRouter
impl Unpin for ModelRouter
impl UnsafeUnpin for ModelRouter
impl UnwindSafe for ModelRouter
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
Source§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more