Expand description
Cost-aware LLM routing: pricing tables, USD budgets, task-class routing.
model-router is the extracted, dependency-light core of the router built
for clawdius. It contains only pure
types and math:
ModelPricing— per-model input/output rates in USD per 1M tokens.PricingTable— a refreshable set ofModelPricingentries with freshness stamps, newer-winsPricingTable::merge, and JSON loading (PricingTable::from_json, featurejson) so rates can be updated at runtime without a crate release.CostTracker— thread-safe accumulation of spend against an optional USD budget; rejects any single record that would push spend over budget.TaskClass— a small generic routing class (Fast/Balanced/Power/Embedding).RoutingRule+Router— task class to model selection with a linked fallback chain, plus quality-tier-aware selection byTaskComplexity.
§Design
The crate is deliberately free of any HTTP client, provider SDK, async runtime, or tool-execution concerns. It answers exactly one question:
given a task class (and optionally a complexity requirement), which model should I call, what is the fallback order, and what did it cost?
Wiring the selected model to an actual API is the caller’s job (see the
optional genai feature for a thin companion adapter to the
genai crate).
§Task-class mapping
clawdius used coding-workflow-shaped task types. TaskClass generalizes
them; the suggested mapping is:
| clawdius task type | TaskClass | typical model tier |
|---|---|---|
Think | TaskClass::Fast | cheap / fast |
Summarize | TaskClass::Fast | cheap / fast |
Test | TaskClass::Fast | cheap / fast |
Plan | TaskClass::Balanced | mid tier |
Review | TaskClass::Balanced | mid tier |
Chat | TaskClass::Balanced | mid tier |
Build | TaskClass::Power | strongest |
| (embeddings) | TaskClass::Embedding | embedding models |
§Feature flags
serde—Serialize/Deserializeon the public data types.json—PricingTable::from_json/PricingTable::to_jsonfor the price-refresh workflow (impliesserde).genai— companion adapter fromgenai::chat::ModelIdento pricing entries (pulls in thegenaicrate; core stays genai-free).
§Guarantees
#![forbid(unsafe_code)]— no unsafe anywhere in the crate.#![deny(missing_docs)]— the entire public API is documented.CostTracker::recordchecks the budget before mutating, so an over-budget request never lands: tracked spend can never exceed the limit (checked-then-added semantics).
Structs§
- Cost
Report - Snapshot of all tracked usage, including budget status.
- Cost
Tracker - Accumulates token usage and USD spend, optionally against a budget.
- Model
Cost Breakdown - Per-model cost breakdown for reporting.
- Model
Pricing - Per-model pricing information.
- Pricing
Table - A refreshable set of
ModelPricingentries keyed by model identifier. - Route
Decision - The result of routing a task: the primary model plus its fallback order.
- Router
- Cost-aware task router.
- Routing
Rule - A routing rule: which model to use for a
TaskClass, with an optional fallback rule tried if the primary is unavailable.
Enums§
- Router
Error - Errors returned by the routing core.
- Task
Class - Generic routing class for a unit of work.
- Task
Complexity - Task complexity levels for cost-aware routing.
Functions§
- default_
pricing_ table - Built-in pricing table for popular models.