Skip to main content

Module eval

Module eval 

Source
Expand description

The corpus fold: joins recorded verdicts to observed build outcomes and reports how trustworthy the verdict is. The fingerprint pairs the two; this is a plain fold over the paired rows, run as needed, not a stored subsystem.

Structs§

CorpusEntry
One paired row: a verdict, its fingerprint, and the outcome it was meant to predict. coverage is the producing binary’s claims, so the fold routes bug-vs-gap against the row, not the running binary.
EvalReport
The fold’s output.
ForecastEntry
One paired row: what triage predicted for a candidate and how the apply went.
ForecastMiss
Observed conflicting paths the forecast did not predict, the worklist row. Path-level, so a pick whose forecast called one path and missed another still surfaces the missed path.
ForecastReport
The forecast fold’s output. The rates are entry-level: a hit is an entry that predicted any conflict and observed any conflict; which paths matched is graded by path_overlap and false_negatives. Forecasts are computed at triage against the then-current target tip, so false negatives partly measure tip drift, not forecast error.
ForecastedConflict
An apply conflict a forecast predicts: one path, its conflict kind. Normalized to RelativePath by the producer, unlike PredictedConflict, which keeps a lossless PathBuf.
MissedBreak
A break the verdict did not flag.
PredictedConflict
A risky apply conflict a verdict predicts: one path, its conflict kind. The path is the reason’s own PathBuf, never dropped.
Ratio
A measured rate kept as hit/total so the sample size is never lost behind a bare percentage. hits <= total always: every mutator grows total by at least as much as hits in the same step.

Enums§

BreakAttribution
Who is responsible for a break the verdict did not flag: the resolver (it checked the class and still missed it), a coverage gap (it never checked the class), or unknown (the row recorded no coverage, or the break carried no class).
BuildOutcome
What the build did once the pick landed. Only CompilationFailed carries a class: an apply conflict and a test regression are not symbol-class breaks. The class is the symbol that actually broke, which the consumer reads from the build error, not from the verdict.
ObservedApply
How the pick actually applied. OutOfBand landings correlate with hard conflicts, so dropping them would bias recall toward easy picks.

Functions§

evaluate_corpus
Fold the paired corpus into accuracy rates and a missed-break list.
evaluate_forecasts
Fold paired forecast rows into accuracy rates and a missed-conflict worklist.
paths_match
Whether two paths recorded relative to different roots name the same file: equal, or one is a /-boundary suffix of the other. Many-to-one by design: a bare Makefile matches any path ending in /Makefile.