model-router
Cost-aware LLM routing core: per-model pricing tables, USD budget tracking, and task-class routing with fallback chains.
This is the extracted, dependency-light routing core of clawdius: pure types and math, no HTTP client, no provider SDK, no async runtime. It answers one question:
given a task (and optionally a complexity requirement), which model should I call, what is the fallback order, and what did it cost?
use ;
let mut router = new;
router.add_rule;
// Route with a stable fallback chain.
let decision = router.route;
assert_eq!;
assert_eq!;
// Estimate cost, then track it against a USD budget.
let pricing = router.pricing_for.unwrap;
println!;
let tracker = new; // $5.00 budget
tracker.record.unwrap;
assert!;
println!;
What's inside
| Type | Purpose |
|---|---|
[ModelPricing] |
Input/output rates in USD per 1M tokens, context window, quality tier, optional updated_at_unix freshness stamp. |
[PricingTable] |
Refreshable pricing set: newer-wins merge, JSON from_json/to_json (feature json). |
[default_pricing_table] |
Built-in rates for popular Claude / GPT / Gemini / GLM / DeepSeek / OpenRouter models (May 2025). |
[CostTracker] |
Thread-safe spend accumulator; checked-then-added budget enforcement: an over-budget record is rejected and nothing is accumulated, so tracked spend can never exceed the limit. |
[TaskClass] |
Generic routing class: Fast, Balanced, Power, Embedding. |
[TaskComplexity] |
Simple → Critical; picks the cheapest quality tier that qualifies. |
[RoutingRule] / [Router] |
Task class → model, with a linked fallback chain and deterministic quality-per-dollar selection. |
Mapping clawdius task types to TaskClass
clawdius used coding-workflow-shaped task types. The generic mapping:
| clawdius task type | TaskClass |
typical model tier |
|---|---|---|
Think |
Fast |
cheap / fast |
Summarize |
Fast |
cheap / fast |
Test |
Fast |
cheap / fast |
Plan |
Balanced |
mid tier |
Review |
Balanced |
mid tier |
Chat |
Balanced |
mid tier |
Build |
Power |
strongest |
| (embeddings) | Embedding |
embedding models |
TaskClass::for_phase_name implements this mapping for clawdius phase
vocabulary ("think", "plan", "build", "implement", "code", "test",
"verify", "review", "reflect", "summarize"), with unknown names
falling back to Balanced.
Keeping prices fresh
Static price tables go stale. ModelPricing entries carry an optional
updated_at_unix stamp (Unix seconds; None for the built-in defaults, i.e.
"freshness unknown"), and PricingTable supports refreshing at runtime
without waiting for a crate release:
-
Publish: stamp entries and serialize a table with the
jsonfeature:use ; let mut table = with_defaults; table.insert; let json = table.to_json.unwrap; // host this at a URL / write to a file -
Consume: load the JSON and merge it over your current table. Entries that are missing or stamped newer replace what you have; everything else is left untouched (so a stale snapshot can never clobber fresh data):
use ; let fresh = from_json?; // or from disk: PricingTable::from_json(&std::fs::read_to_string("pricing.json")?)? let mut router = new; router.merge_pricing; // newer-or-missing entries win
The JSON shape is a plain object mapping model id to pricing fields:
hostable as-is, and snapshots without updated_at_unix (e.g. pre-0.1.1
exports) still load:
Features
serde(off by default):Serialize/Deserializeon the public data types.json(off by default, impliesserde):PricingTable::from_json/PricingTable::to_jsonfor the refresh workflow above.genai(off by default): thin companion adapter mappinggenaiModelIdenvalues to pricing entries (model_router::genai_compat). The core never depends on genai.
Other LLM clients work just as well: model keys are opaque strings, so any client's model names can be used directly as pricing-table keys.
Guarantees & precision notes
#![forbid(unsafe_code)]and#![deny(missing_docs)].- Routing is deterministic: the pricing table is a
BTreeMapand selection sorting is stable, so ties resolve to the alphabetically first model. CostTrackeraccumulates in fixed point at $0.0001 granularity, truncating: recorded deltas never exceed the true cost, totals err slightly low, and costs under $0.0001 truncate to zero. Budget enforcement runs inf64before any state is mutated.
Testing
- 18 unit tests ported/derived from clawdius's
model_router.rs. - 16 price-freshness tests:
updated_at_unixstamping, merge semantics (newer wins, missing added, stale preserved, unstamped never overwrites),Router::merge_pricing, JSON roundtrip / legacy-JSON loading / error cases, and an end-to-end refresh workflow (the JSON ones run with--features json). - Property-based tests (
proptest): cost accumulation monotonicity, budget never overshot when all spend flows throughrecord, record/estimate consistency, and stable fallback ordering. - Run:
cargo test(add--features jsonfor the JSON refresh tests and--features genaifor the adapter tests).
Stability
model-router is 1.0. Breaking changes require a minor bump and a #[deprecated]
window of at least one minor release; MSRV bumps are minor releases, never
patch releases. missing_docs is denied and unsafe is forbidden. See the
changelog for the short list of explicitly non-contractual behaviours.
License
Apache-2.0 (matching clawdius).