Crate tokenmiser_router 

Classify prompt difficulty and pick a model.

Tier 0 (tier0.rs) is a sub-microsecond heuristic over length, keyword, role and JSON-mode signals. Tier 1 (tier1.rs) is an exemplar-based semantic classifier sharing the L2 cache’s bge-small embedder, used when Tier 0 is not decisive.

Re-exports

pub use dsl::PolicyEngine;

pub use dsl::RequestView;

pub use policy::RoutingPolicy;

pub use policy::RoutingTarget;

pub use replay::replay;

pub use replay::ReplayResult;

pub use tier0::tier0_difficulty;

pub use tier1::Tier1Classifier;

pub use tier2::should_escalate;

pub use tier2::CascadeConfig;

pub use tier2::EscalateDecision;

Modules

dsl

Rhai-scripted routing policy.

policy

Maps a Difficulty to a concrete (provider, model) target.

replay

Backs tokenmiser policy test: replays a JSONL request log through a candidate Rhai policy and tallies the projected routing distribution.

tier0

Tier 0 heuristic prefilter over prompt length, keywords and the JSON-mode flag. Classifies the confident extremes so Tier 1 only runs on the ambiguous middle band; anything undecided falls through as Medium.

tier1

Tier 1 semantic classifier: difficulty exemplars are embedded at startup with the same bge-small-en-v1.5 model the L2 cache loads, and a request takes the difficulty of its nearest exemplar.

tier2

Speculative cascade: a runtime mode, not a classifier. The cheap model always runs first and only low confidence escalates to the frontier.

Structs

RouteDecision

What model to call, why, and the counterfactual model used for cost accounting and shadow A/B.

Router

The orchestrator that runs Tier 0 → Tier 1 → policy in order.

Enums

Difficulty

Coarse difficulty class used by the policy to pick a model.

RouteTier