pub struct Beat {
pub pillar: Option<u8>,
pub incumbent: String,
pub incumbent_pinned: Option<String>,
pub canonical_task: Option<String>,
pub metric: String,
pub direction: String,
pub baseline_value: Option<f64>,
pub baseline_floor: Option<f64>,
pub beat_threshold: Option<f64>,
pub baseline_sourced_date: Option<String>,
pub approved_compute: Option<String>,
pub ci_gate_name: String,
}Expand description
Parameters of a head-to-head BEAT benchmark (metadata.kind: beat-benchmark,
PMAT-741): a falsifiable, CI-wired claim that aprender meets-or-beats an
incumbent (scikit-learn / PyTorch / Unsloth / Ollama·llama.cpp) on the
incumbent’s own canonical task — the measurement backbone of the four-pillar
“replace AND beat” mission. Required-shape is enforced by
validate_beat_benchmark in the validator (BEAT-001..007).
Fields§
§pillar: Option<u8>Which pillar (1=sklearn, 2=PyTorch, 3=Unsloth, 4=Ollama/llama.cpp).
incumbent: StringThe incumbent being beaten — must name one of the four pillars.
incumbent_pinned: Option<String>How/when the baseline was pinned (free-form provenance).
canonical_task: Option<String>The canonical task on which the beat is measured (apples-to-apples).
metric: StringThe measured metric (e.g. accuracy, wall_clock_ms, tokens_per_sec, mse).
direction: Stringhigher_is_better or lower_is_better — fixes the regression direction.
baseline_value: Option<f64>The incumbent’s pinned baseline value.
baseline_floor: Option<f64>Optional worst-case incumbent value (e.g. sklearn min over seeds).
beat_threshold: Option<f64>The threshold aprender must meet/beat; CI fails on regression past it.
baseline_sourced_date: Option<String>When the baseline was sourced (ISO date).
approved_compute: Option<String>CPU or GPU — the compute approved for this gate.
ci_gate_name: StringThe CI test/gate name that enforces this beat.
Implementations§
Source§impl Beat
impl Beat
Sourcepub fn evaluate(&self, measured: f64) -> Option<BeatOutcome>
pub fn evaluate(&self, measured: f64) -> Option<BeatOutcome>
Evaluate a measured value against this beat’s pinned beat_threshold,
honoring direction:
higher_is_better⇒Woniffmeasured >= beat_thresholdlower_is_better⇒Woniffmeasured <= beat_threshold
Returns None when the contract is too malformed to judge (no
beat_threshold, a non-finite threshold/measurement, or an unknown
direction) — the caller should treat that as a hard error, not a pass.
The validator’s BEAT-004/BEAT-005 rules reject such contracts up front,
so a well-formed contract always yields Some.