Struct CaseResult 

pub struct CaseResult {

    pub id: String,
    pub tags: Vec<String>,
    pub samples: u32,
    pub passes: u32,
    pub pass_rate: f64,
    pub pass: bool,
    pub flaky: bool,
    pub reason: String,
    pub cost: Option<f64>,
    pub latency_ms: u64,
    pub turns: Option<u64>,
    pub cache_hits: u32,
    pub cache_misses: u32,
    pub samples_detail: Vec<SampleVerdict>,
    pub answer_dist: Option<BTreeMap<String, u32>>,
}

A case’s aggregate result over samples repeated runs (pass-rate methodology). At samples == 1 every field is bit-identical to the pre-repeated-sampling report shape — that identity is the whole back-compat story, both for the gate’s case-level comparison and for the Pareto table.

Fields

id: String

tags: Vec<String>

samples: u32

How many times this case was sampled. 0 only on a report predating repeated sampling — the sentinel crate::Report::backfill_legacy detects and migrates forward. Every freshly-produced result sets this to samples.max(1).

passes: u32

How many of samples passed.

pass_rate: f64

passes as f64 / samples as f64. Kept as a stored field (not recomputed on read) so a legacy report’s backfilled value round-trips identically to a freshly-computed one.

pass: bool

Fully passing: every sample passed. At samples == 1 this is exactly today’s single-run pass flag.

flaky: bool

Some samples passed, some failed (0 < passes < samples). Always false at samples <= 1.

reason: String

A representative reason: the modal failure reason among failing samples, or "match" when every sample passed.

cost: Option<f64>

Summed over samples — repeated sampling multiplies spend by samples, so the total is what a run’s cost column should reflect, not a per-sample average. None when any sample’s backend couldn’t report cost — absent is not zero, so the sum doesn’t quietly understate.

latency_ms: u64

Averaged over samples.

turns: Option<u64>

Averaged over samples (num_turns) — the round-trip diagnostic. None when any sample’s backend couldn’t report a turn count.

cache_hits: u32

Samples served from the trace cache (re-scored, 0 LLM) vs freshly re-run this invocation.

cache_misses: u32

samples_detail: Vec<SampleVerdict>

Per-sample verdicts. Empty on a legacy report (there is nothing to backfill this from).

answer_dist: Option<BTreeMap<String, u32>>

Distinct-answer distribution across samples. Some only when the run recorded answers (--record-answers) — absence is unambiguous (not just an empty map).

Implementations

impl CaseResult

pub fn output_stability(&self) -> OutputStability

Classify this case’s output-level stability from its own samples/answer_dist — see OutputStability for what each variant means and why Unstable is a labelled upper bound rather than an exact count.

Trait Implementations

impl Clone for CaseResult

fn clone(&self) -> CaseResult

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for CaseResult

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for CaseResult

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Serialize for CaseResult

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.