Skip to main content

Module model_router

Module model_router 

Source
Expand description

Intent-based model routing (P8 / DIM 3 — Leistungsstufe).

Classifies each request’s last user message via crate::core::intent_engine and resolves the resulting a model tier to a concrete routing target using [proxy.routing.tiers]. The forward path then rewrites the request body’s model field (and optionally re-targets the upstream provider).

Fail-open by construction: any classification miss, absent tier key, or empty target string leaves the request untouched. Premium work is never silently downgraded unless the operator explicitly configures a tier target.

Opt-in only: requires [proxy.routing] enabled = true AND at least one tier entry. Without both, this module is a no-op.

§Interaction with other routing mechanisms

  • Aliases (exact model-name swap) run first — if the requested model matches an alias, the aliased target is used and tier routing is skipped.
  • Policy gate (model ceiling, budgets) runs after routing — it sees the post-routing model and can veto it.
  • Effort routing (thinking budget) is orthogonal — it adjusts the reasoning_effort / thinking parameter, not the model identity.

§Cost-quality awareness

When live model prices are available (loaded by the proxy at startup from ~/.config/lean-ctx/model-prices.json), the router annotates its decision with cost savings estimates. This is observability only — the tier lookup is the authoritative routing decision, not a dynamic cost optimizer.

Structs§

RoutingDecision
A routing decision record — emitted for observability and future OCLA bus integration (P2). Deterministic: same input → same decision (#498).

Functions§

apply_decision
Applies a routing decision to a mutable request body (in-place model swap).
classify_only
Classifies a request without applying routing — for dry-run / observability.
route
Applies intent-based tier routing to a request body.