Struct BenchRunner 

pub struct BenchRunner { /* private fields */ }

Top-level orchestrator for benchmark execution.

Manages the full lifecycle of a benchmark run:

1. Loading workloads (built-in or from file)
2. Cost estimation and budget enforcement
3. Warm-up phase (iterations discarded)
4. Measurement phase (iterations recorded)
5. Concurrent execution and sweep modes
6. Metric aggregation and CV warnings
7. Baseline save and regression detection

Implementations

impl BenchRunner

pub fn new(config: BenchConfig) -> Self

Creates a new BenchRunner with the given configuration.

Initializes BaselineStore at the configured baseline path and a default CostTracker with standard model pricing.

pub async fn run(&self) -> Result<Vec<BenchmarkResult>>

Runs the full benchmark suite and returns results.

Execution Flow

1. Resolves workloads (specific workload via --workload or all built-in)
2. Estimates cost and enforces budget (dry-run, max-cost-usd, confirm-cost)
3. For each workload: a. Warm-up phase: run config.warmup iterations, discard results b. Measurement phase: run config.runs iterations, collect metrics c. If sweep mode: iterate through concurrency levels d. If concurrency > 1: spawn concurrent tasks
4. Aggregates metrics and emits CV warnings
5. Returns collected BenchmarkResult values

Errors

• BenchError::WorkloadNotFound if a specified workload file doesn’t exist
• BenchError::Baseline if cost exceeds --max-cost-usd

pub fn save_baseline(&self, results: &[BenchmarkResult]) -> Result<()>

Saves current results as the regression baseline.

Persists metrics via BaselineStore for later regression detection.

pub fn check_regression( &self, results: &[BenchmarkResult], ) -> Result<Vec<RegressionReport>>

Checks results against saved baseline using configured tolerance.

For benchmark timing metrics, a regression means the current value is higher than the baseline (worse performance). The formula is:

regression detected when (current - baseline) / baseline > tolerance

Returns a list of RegressionReport entries for any metrics that exceed the tolerance threshold. An empty list means no regressions.

Exit Code Contract

The CLI layer (Task 8.1) should exit with code 2 when this method returns a non-empty list, and exit with code 0 otherwise.