Skip to main content

Crate model_router

Crate model_router 

Source
Expand description

Cost-aware LLM routing: pricing tables, USD budgets, task-class routing.

model-router is the extracted, dependency-light core of the router built for clawdius. It contains only pure types and math:

  • ModelPricing — per-model input/output rates in USD per 1M tokens.
  • PricingTable — a refreshable set of ModelPricing entries with freshness stamps, newer-wins PricingTable::merge, and JSON loading (PricingTable::from_json, feature json) so rates can be updated at runtime without a crate release.
  • CostTracker — thread-safe accumulation of spend against an optional USD budget; rejects any single record that would push spend over budget.
  • TaskClass — a small generic routing class (Fast / Balanced / Power / Embedding).
  • RoutingRule + Router — task class to model selection with a linked fallback chain, plus quality-tier-aware selection by TaskComplexity.

§Design

The crate is deliberately free of any HTTP client, provider SDK, async runtime, or tool-execution concerns. It answers exactly one question:

given a task class (and optionally a complexity requirement), which model should I call, what is the fallback order, and what did it cost?

Wiring the selected model to an actual API is the caller’s job (see the optional genai feature for a thin companion adapter to the genai crate).

§Task-class mapping

clawdius used coding-workflow-shaped task types. TaskClass generalizes them; the suggested mapping is:

clawdius task typeTaskClasstypical model tier
ThinkTaskClass::Fastcheap / fast
SummarizeTaskClass::Fastcheap / fast
TestTaskClass::Fastcheap / fast
PlanTaskClass::Balancedmid tier
ReviewTaskClass::Balancedmid tier
ChatTaskClass::Balancedmid tier
BuildTaskClass::Powerstrongest
(embeddings)TaskClass::Embeddingembedding models

§Feature flags

  • serde — Serialize/Deserialize on the public data types.
  • json — PricingTable::from_json / PricingTable::to_json for the price-refresh workflow (implies serde).
  • genai — companion adapter from genai::chat::ModelIden to pricing entries (pulls in the genai crate; core stays genai-free).

§Guarantees

  • #![forbid(unsafe_code)] — no unsafe anywhere in the crate.
  • #![deny(missing_docs)] — the entire public API is documented.
  • CostTracker::record checks the budget before mutating, so an over-budget request never lands: tracked spend can never exceed the limit (checked-then-added semantics).

Structs§

CostReport
Snapshot of all tracked usage, including budget status.
CostTracker
Accumulates token usage and USD spend, optionally against a budget.
ModelCostBreakdown
Per-model cost breakdown for reporting.
ModelPricing
Per-model pricing information.
PricingTable
A refreshable set of ModelPricing entries keyed by model identifier.
RouteDecision
The result of routing a task: the primary model plus its fallback order.
Router
Cost-aware task router.
RoutingRule
A routing rule: which model to use for a TaskClass, with an optional fallback rule tried if the primary is unavailable.

Enums§

RouterError
Errors returned by the routing core.
TaskClass
Generic routing class for a unit of work.
TaskComplexity
Task complexity levels for cost-aware routing.

Functions§

default_pricing_table
Built-in pricing table for popular models.