pub struct CaseKey<'a> {
pub case_id: &'a str,
pub question: &'a str,
pub agent_sha: &'a str,
pub model: &'a str,
pub context_sha: &'a str,
pub sample: u32,
pub backend: Backend,
pub max_turns: Option<u32>,
pub tier_binding: Option<&'a str>,
}Expand description
The inputs that determine a case’s result. Everything here is in the cache key; the golden’s expectation (expected/tolerance/match) deliberately is not, which is what makes re-scoring a changed expectation a cache hit.
Fields§
§case_id: &'a str§question: &'a str§agent_sha: &'a str§model: &'a strThe whole-run --model binding, or FRONTMATTER_MODEL on the ablation path.
context_sha: &'a str§sample: u32Which repeated-sample run this is (0-indexed). Distinct samples of the same case are distinct cache entries — the pass-rate methodology depends on each sample actually invoking the agent (or replaying its OWN prior trace), not silently collapsing onto sample 0.
backend: BackendWhich back-end/runtime this run is measuring. Part of the key so runs on different back-ends can never hit each other’s cache.
max_turns: Option<u32>The --max-turns cap this run was invoked with, or None when the back-end’s own default
applied. Part of the key for the same reason backend is: a run capped to 1 turn and a run
left at the default turn budget are different experiments and must never share a cache
entry. None keeps the pre-existing (no-such-knob) key shape untouched — see canonical()
below, which appends this field to the key array only when it is Some.
tier_binding: Option<&'a str>A canonical, self-describing rendering of a differentiated --strong/--cheap/
--orchestrator (or --models-config) binding (e.g.
"strong=sonnet,cheap=haiku,orchestrator=sonnet"), or None on every pre-existing path —
the single-model --models sweep and the ablation/frontmatter path both leave this unset,
since model alone already determines what ran there. Part of the key (decision 68: any
parameter that shapes how a run executes belongs in the cache key) for the same reason
backend/max_turns are: a differentiated binding is a different experiment from any
single-model run and must never share a cache entry with one, even when model happens to
equal one of the three tier values. None keeps the pre-existing key shape untouched — see
canonical() below, which appends this field to the key array only when it is Some.