Expand description
Reproducible scorecard (#211): one command → compression savings + retrieval recall/MRR + latency over a fixed, deterministic scenario matrix.
The quality metrics (savings, recall, MRR) are reproducible run-to-run and
machine-to-machine because the corpus is generated deterministically and the
retrieval path is pure BM25. Latency is measured wall-clock and therefore
reported but not part of the determinism contract (see determinism_digest).
Modules§
- dual_
arm - Dual-arm self-verify bench (#361 Phase 3): prove the billed advantage of a long-lived, cache-aware lean-ctx session over a phase-isolated one — locally, deterministically, before any external (tokbench) run.
Structs§
- Aggregate
- Cross-scenario averages.
- Scenario
Score - Per-scenario result.
- Scorecard
- The full scorecard.
Functions§
- run_
scorecard - Run the full scenario matrix and assemble the scorecard.