Skip to main content

run_variant_benchmark

Function run_variant_benchmark 

Source
pub fn run_variant_benchmark(
    variant: &str,
    cmd: &str,
    min_files: f64,
) -> Result<AgentBenchSummary, String>
Expand description

Wave-15 variant benchmark: same protocol as run_agent_benchmark over the full benchmarks/tasks.json corpus, recording the variant name in the summary. cmd is a shell command; the variant context artifact is expected to be generated inside it (see benchmarks/run_agent_bench.sh).