pub fn aggregate_benchmark(
iteration_dir: &Path,
skill_name: &str,
) -> Result<BenchmarkData>Expand description
Aggregate pass-rate / time / token stats for the configurations found in an eval workspace iteration directory.
Layout expected (per skill-creator SKILL.md):
<iteration-dir>/
<eval-name>/
with_skill/{grading.json, timing.json, outputs/...}
without_skill/{grading.json, timing.json, outputs/...} # or old_skill/
eval_metadata.jsonwith_skill is the treatment; the first of without_skill / old_skill
that exists is the baseline. Missing files are skipped (the eval is still
counted, but contributes no data to that configuration).