Expand description
Routing configuration (SPEC §8.4) — declarative, agent-first.
Config is the policy an operator (or agent) hands Firstpass: an ordered list of routes,
each matching a slice of traffic to a mode, a model ladder, and gates; plus budget caps
and escalation rules. The first matching route wins, so specific routes go first and a
bare match = {} catch-all goes last.
Structs§
- Adaptive
Config - Config for online/adaptive conformal serving (
crate::conformal::AdaptiveConformal). - Budget
- Spend caps.
Nonemeans “no cap”. - Config
- Top-level configuration document.
- Escalation
- Escalation limits and session-level promotion.
- GateDef
- A user-defined gate (SPEC §8.1). Exactly one kind per definition:
- Judge
Def - Configuration for a native LLM-judge gate (SPEC §8.3): a separate model grades the candidate against a rubric. The runner enforces maker ≠ checker (a model never grades its own output) and treats the candidate as data, not instructions.
- Match
- Predicate over a request’s
Features. Every present field is an AND-constraint; absent fields are wildcards. - Model
Ref - A parsed
provider/modelreference. - Provider
Def - A model provider a ladder can route to. Declared as
[[provider]]in TOML; referenced from a ladder as<id>/<model>(e.g.groq/llama-3.3-70b-versatile). - Route
- One routing rule.
- Session
Promotion - “After N failures within a window, promote this session’s starting rung.”
Enums§
- Dialect
- The wire API a provider speaks.
anthropic= Messages API;openai= Chat Completions API (the de-facto standard that nearly every hosted and open-source model host implements). - Mode
- Serving mode for a route (SPEC glossary).
- OnExhausted
- What to do when a spend cap is hit (SPEC §8.4 — never brick the customer).
- String
OrVec - A field that accepts either a single string or a list of strings in TOML/JSON.