pub struct Comparison {
pub baseline_suite: String,
pub current_suite: String,
pub headline: Headline,
pub excluded: Vec<ExcludedItem>,
pub changes: Vec<Change>,
}Expand description
What changed between two runs of the same suite.
The field order is the reading order, and it is deliberate: the headline (or
the refusal to produce one) and the exclusions come before the changes, in
the machine-readable form as much as in Comparison::summary.
Fields§
§baseline_suite: StringThe suite the baseline reported on.
current_suite: StringThe suite the current run reported on.
headline: HeadlineThe figures, or the reason there are none.
excluded: Vec<ExcludedItem>Items that appeared in both runs and could not be paired, with why.
changes: Vec<Change>Every change on the items that stayed paired, in current-run item order followed by removals.
Implementations§
Source§impl Comparison
impl Comparison
Sourcepub fn is_unchanged(&self) -> bool
pub fn is_unchanged(&self) -> bool
Returns true when nothing changed and nothing was excluded.
Sourcepub fn deterministic_regressions(&self) -> Vec<&Change>
pub fn deterministic_regressions(&self) -> Vec<&Change>
The deterministic regressions — behaviour that got worse.
Sourcepub fn signal_regressions(&self) -> Vec<&Change>
pub fn signal_regressions(&self) -> Vec<&Change>
The deterministic regressions that a control run says are larger than the harness’s own variation.
Without a noise floor this is every regression: an
Unmeasured movement is not known to be
noise, and treating the unknown as harmless is how a regression ships.
Sourcepub fn judge_drifts(&self) -> Vec<&Change>
pub fn judge_drifts(&self) -> Vec<&Change>
The judge drifts — the same behaviour, graded differently.
Sourcepub fn derived_input_changes(&self) -> Vec<&Change>
pub fn derived_input_changes(&self) -> Vec<&Change>
The items whose declared-derived parts changed: the intended effect of the change being measured, and not a broken pairing.
Sourcepub fn has_deterministic_regression(&self) -> bool
pub fn has_deterministic_regression(&self) -> bool
Returns true when at least one item’s deterministic assertions got
worse.
Sourcepub fn has_signal_regression(&self) -> bool
pub fn has_signal_regression(&self) -> bool
Returns true when at least one regression exceeds the measured noise
floor. This is the question a merge gate should ask.
Sourcepub fn has_judge_drift(&self) -> bool
pub fn has_judge_drift(&self) -> bool
Returns true when at least one judge score moved beyond the tolerance.
This is a question about the measurement, not about the agent.
Sourcepub fn corpus_defects(&self) -> Vec<&ExcludedItem>
pub fn corpus_defects(&self) -> Vec<&ExcludedItem>
The excluded items whose recorded parts changed.
These are not a result about the model and must never be read as one: something regenerated testimony about what the system actually emitted, which is a defect in the corpus or in the tooling that touched it. They are excluded from every figure, and this is where they are named.
Sourcepub fn has_corpus_defect(&self) -> bool
pub fn has_corpus_defect(&self) -> bool
Returns true when a recorded part changed anywhere in the corpus.
The fraction of the paired corpus that could not be compared.
Sourcepub fn is_withheld(&self) -> bool
pub fn is_withheld(&self) -> bool
Returns true when there is no headline figure to read, because too much
of the corpus went unpaired.