pub struct ComplexityRoutingConfig {
pub triage_provider: Option<ProviderName>,
pub bypass_single_provider: bool,
pub tiers: TierMapping,
pub max_triage_tokens: u32,
pub triage_timeout_secs: u64,
pub fallback_strategy: Option<String>,
}Expand description
Configuration for complexity-based triage routing (routing = "triage").
When [llm] routing = "triage" is set, a cheap triage model classifies each request
and routes it to the appropriate tier provider. Requires at least one tier mapping.
§Example
[llm]
routing = "triage"
[llm.complexity_routing]
triage_provider = "local-fast"
[llm.complexity_routing.tiers]
simple = "local-fast"
medium = "haiku"
complex = "sonnet"
expert = "opus"Fields§
§triage_provider: Option<ProviderName>Provider name from [[llm.providers]] used for triage classification.
bypass_single_provider: boolSkip triage when all tiers map to the same provider.
tiers: TierMappingTier-to-provider name mapping.
max_triage_tokens: u32Max output tokens for the triage classification call. Default: 50.
triage_timeout_secs: u64Timeout in seconds for the triage classification call. Default: 5. On timeout, falls back to the default (first) tier provider.
fallback_strategy: Option<String>Optional fallback strategy when triage misclassifies.
Only "cascade" is currently supported (Phase 4).
Trait Implementations§
Source§impl Clone for ComplexityRoutingConfig
impl Clone for ComplexityRoutingConfig
Source§fn clone(&self) -> ComplexityRoutingConfig
fn clone(&self) -> ComplexityRoutingConfig
Returns a duplicate of the value. Read more
1.0.0 · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
Performs copy-assignment from
source. Read moreSource§impl Debug for ComplexityRoutingConfig
impl Debug for ComplexityRoutingConfig
Source§impl Default for ComplexityRoutingConfig
impl Default for ComplexityRoutingConfig
Source§impl<'de> Deserialize<'de> for ComplexityRoutingConfig
impl<'de> Deserialize<'de> for ComplexityRoutingConfig
Source§fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
Deserialize this value from the given Serde deserializer. Read more
Auto Trait Implementations§
impl Freeze for ComplexityRoutingConfig
impl RefUnwindSafe for ComplexityRoutingConfig
impl Send for ComplexityRoutingConfig
impl Sync for ComplexityRoutingConfig
impl Unpin for ComplexityRoutingConfig
impl UnsafeUnpin for ComplexityRoutingConfig
impl UnwindSafe for ComplexityRoutingConfig
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
Source§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§impl<T> IntoRequest<T> for T
impl<T> IntoRequest<T> for T
Source§fn into_request(self) -> Request<T>
fn into_request(self) -> Request<T>
Wrap the input message
T in a tonic::Request