Struct BaselineComparison 

pub struct BaselineComparison {
    pub dataset: String,
    pub model: String,
    pub run_id_memory_on: String,
    pub run_id_memory_off: String,
    pub deltas: Vec<ScenarioDelta>,
    pub aggregate_delta: f64,
}

Comparison between two benchmark runs (memory-on vs memory-off).

Use BaselineComparison::compute to join two BenchRuns by scenario ID and compute per-scenario deltas and an aggregate mean delta.

Examples

use zeph_bench::{BenchRun, RunStatus, ScenarioResult, Aggregate};
use zeph_bench::baseline::BaselineComparison;

fn make_run(run_id: &str, scores: &[(&str, f64)]) -> BenchRun {
    BenchRun {
        dataset: "test".into(),
        model: "model".into(),
        run_id: run_id.into(),
        started_at: "2026-01-01T00:00:00Z".into(),
        finished_at: "2026-01-01T00:01:00Z".into(),
        status: RunStatus::Completed,
        results: scores.iter().map(|(id, score)| ScenarioResult {
            scenario_id: id.to_string(),
            score: *score,
            response_excerpt: String::new(),
            error: None,
            elapsed_ms: 0,
        }).collect(),
        aggregate: Aggregate::default(),
    }
}

let on = make_run("r1", &[("s1", 1.0), ("s2", 0.5)]);
let off = make_run("r2", &[("s1", 0.5), ("s2", 0.0)]);
let cmp = BaselineComparison::compute(&on, &off);
assert_eq!(cmp.deltas.len(), 2);
assert!((cmp.aggregate_delta - 0.5).abs() < f64::EPSILON);

Fields

dataset: String

Dataset name (from the memory-on run).

model: String

Model identifier (from the memory-on run).

run_id_memory_on: String

Run ID of the memory-on run.

run_id_memory_off: String

Run ID of the memory-off run.

deltas: Vec<ScenarioDelta>

Per-scenario deltas, sorted by scenario_id.

Only scenarios present in both runs are included (inner join).

aggregate_delta: f64

Arithmetic mean of all delta values. 0.0 if no scenarios overlap.

Implementations

impl BaselineComparison

pub fn compute(memory_on: &BenchRun, memory_off: &BenchRun) -> Self

Compute deltas by joining memory_on and memory_off runs on scenario_id.

Only scenarios present in both runs are included. Non-overlapping scenarios are silently dropped. aggregate_delta is the arithmetic mean of all per-scenario deltas; 0.0 when there are no overlapping scenarios.

pub fn write_comparison_json(&self, output_dir: &Path) -> Result<(), BenchError>

Write this comparison as pretty-printed JSON to {output_dir}/comparison.json.

The file is written atomically via a .tmp sibling + rename, so a concurrent SIGINT cannot leave a half-written file.

Errors

Returns BenchError::InvalidFormat on serialization failure and BenchError::Io on write failure.

pub fn write_delta_table(&self, summary_path: &Path) -> Result<(), BenchError>

Append a delta table section to the Markdown file at summary_path.

Creates the file if it does not exist. The section header is ## Baseline Comparison (Memory On vs Off) followed by a Markdown table of per-scenario deltas and a final aggregate delta line.

Errors

Returns BenchError::Io on read/write failure.

Trait Implementations

impl Clone for BaselineComparison

fn clone(&self) -> BaselineComparison

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for BaselineComparison

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for BaselineComparison

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Serialize for BaselineComparison

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.