Struct Judge 

pub struct Judge {
    pub iterations: u32,
    pub warmup: u32,
    pub size_plan_override: Option<Vec<(OpKind, OpSize)>>,
    pub fixtures: Option<HashMap<(OpKind, DType, SizeClass), Vec<CorrectnessFixture>>>,
}

The runner. Consumes a ProbeReport (so it knows which devices exist and which equivalence classes to share across), builds the full measurement matrix, and produces a ProfileReport.

Fields

iterations: u32

warmup: u32

size_plan_override: Option<Vec<(OpKind, OpSize)>>

Optional shrunk size ladder. None = use the full default ladder: three square/general sizes per op (up to 1024×1024 matmul / 2²⁰ elementwise), PLUS decode-shaped cells on the decode-attention path (seq_q=1 matmul GEMV; wide-k_len MaxReduce/SumReduce; decode-score-row Sub/Exp/Div). Tests supply a shrunk ladder to stay fast.

fixtures: Option<HashMap<(OpKind, DType, SizeClass), Vec<CorrectnessFixture>>>

Optional pre-validated correctness fixtures keyed by (op, dtype, size_class). When a fixture covers a profiling cell, the Judge validates every CellRun’s output against it via validate_against_fixture and assigns each ProfileEntry::max_rel_error from the verdict, bypassing the inline pairwise-consensus path for that cell. Cells without a matching fixture fall back to the consensus path (today’s default behavior on multi-backend systems).

One key may map to multiple fixtures when capture used different input_seeds; the Judge applies the first whose fixture-shape and the cell’s regenerated input agree. If no fixture matches the cell’s input seed (mismatch logged), the fallback consensus path is used for the cell.

Implementations

impl Judge

pub fn with_fixtures_from(path: &Path) -> Result<Self>

Construct a Judge with the default measurement parameters and the fixture set loaded from path. The path may point at a single *.json file or a directory; in the directory case every *.json under the root is loaded recursively (the fuel-correctness-fixtures capture binary writes v1/<dtype>/<op>.json, so a single root covers an entire fixture distribution).

Returns an io::Error wrapped as a fuel-core Error if the path can’t be read or any file fails to parse as a FixtureFile.

Use: pass a fixture root once at Judge construction time; every cell the Judge profiles thereafter validates against the fixture (if one exists for that cell) instead of inline consensus. Cells with no matching fixture fall back to the consensus path.

pub fn run(&self, probe: &ProbeReport) -> ProfileReport

Run the full profile matrix for every equivalence class in probe. Skips backends that aren’t compiled in; logs (stderr) any backend that’s present but hasn’t been wired into the Judge yet (e.g. Vulkan in this revision).

Reference-retirement note (2026-06-07): the loop order is inverted relative to the pre-retirement shape — outer over (op, size), inner over equivalence classes. This lets Judge collect all backends’ outputs at a given cell, compute pairwise consensus across them, and assign each entry’s max_rel_error relative to the consensus cluster (no privileged “reference output” required). When only one backend is available at a cell, consensus is trivial and max_rel_error = 0.0 (no peers to compare against — the caller treats the lone backend as authoritative).

Per-alternative measurement (2026-06-08): for each equivalence-class representative the Judge now walks KernelBindingTable::lookup_alternatives(op_kind, dtypes, backend) and emits one CellRun per registered alternative. One alternative is timed via the standard realizer path; its CellRun is attributed to the sibling the picker ACTUALLY dispatched (the bridge’s post-realize report — Session 3 rider, 2026-06-11 — with first-registered fallback when the plan had no entry for the measured root). The remaining alternatives at the same (op, dtypes, backend) key are timed via a direct kernel-pointer call so AOCL/MKL siblings get distinct latency numbers.

Trait Implementations

impl Default for Judge

fn default() -> Self

Returns the “default value” for a type.