pub struct Report {
pub dataset: Option<String>,
pub context_version: Option<String>,
pub context_injection: Option<ContextInjectionReport>,
pub parallel: usize,
pub selected_cases: usize,
pub total_cases: usize,
pub configs: Vec<ConfigReport>,
pub backend: Backend,
}Expand description
A full eval report (one config per tier→model binding). Serialized by warble eval run --out
and consumed as the baseline / candidate by warble eval gate.
Fields§
§dataset: Option<String>§context_version: Option<String>§context_injection: Option<ContextInjectionReport>Dispatch-time context identity copied from context-report.json. Fingerprints/counts only: the
report never embeds private business-rule text. None for legacy agents.
parallel: usizeConcurrency the cases ran at (1 = serial). Recorded because parallelism can inflate per-case latency (queueing), so latency columns are only comparable at equal levels.
selected_cases: usizeGolden cases scored vs the golden file’s total. selected_cases < total_cases means a
--tags/--sample subset (a smoke run) — recorded so a partial run is never mistaken for
a full one and eval gate can see the comparison was over fewer cases (no silent caps).
Both 0 in a pre-P3 report (the field was absent).
total_cases: usize§configs: Vec<ConfigReport>§backend: BackendWhich back-end/runtime produced this report. #[serde(default)] so a legacy report (no
field at all) deserializes to Backend::default — the back-end that behavior always
meant before this dimension existed — never a hard parse failure.
Implementations§
Source§impl Report
impl Report
Sourcepub fn backfill_legacy(&mut self)
pub fn backfill_legacy(&mut self)
Migrate a report from before repeated sampling to today’s shape. A case with samples == 0
is the legacy sentinel — freshly-produced results always set samples >= 1 — so such a case
is backfilled to a single deterministic sample matching its recorded pass outcome, and each
touched tag’s pass_rate_sum is rebuilt from the (now-backfilled) case pass rates.
One historical distinction is lost in the backfill: whether a legacy case was a cache hit.
The old schema’s single cache_hit bool has no home in the new per-case cache_hits/
cache_misses counts once it’s not in the JSON at all (serde drops unknown fields), so every
legacy case backfills to cache_hits: 0, cache_misses: 1 (as if freshly run). This only
affects cache-visibility reporting on a re-processed old report, never gate correctness —
the gate never looks at cache counts.