Skip to main content

Module report

Module report 

Source
Expand description

Paired report + non-regression gate (#237).

Every task is scored under both conditions, giving a paired sample of per-task deltas (lean_ctx − baseline). From those we compute the mean delta, a deterministic bootstrap 95% confidence interval (fixed seed → byte-identical CI on every machine), win/tie/loss counts and pass-rate deltas, then collapse it all into a single Verdict that drives the CI quality gate.

Structs§

AbReport
The full A/B report: provenance, per-task records, aggregate stats and the verdict.
AbStats
Aggregate statistics over all paired records.
PairRecord
One task scored under both conditions, with the audit digests for each window + answer.
ReportConfig
Knobs for the statistics + gate. Defaults are deterministic and strict.

Enums§

Verdict
The headline conclusion of a run.

Constants§

REPORT_KIND
Report schema discriminator + version (also guards artifact parsing).
REPORT_SCHEMA_VERSION