Skip to main content

Module scorecard

Module scorecard 

Source
Expand description

Reproducible scorecard (#211): one command → compression savings + retrieval recall/MRR + latency over a fixed, deterministic scenario matrix.

The quality metrics (savings, recall, MRR) are reproducible run-to-run and machine-to-machine because the corpus is generated deterministically and the retrieval path is pure BM25. Latency is measured wall-clock and therefore reported but not part of the determinism contract (see determinism_digest).

Modules§

dual_arm
Dual-arm self-verify bench (#361 Phase 3): prove the billed advantage of a long-lived, cache-aware lean-ctx session over a phase-isolated one — locally, deterministically, before any external (tokbench) run.

Structs§

Aggregate
Cross-scenario averages.
ScenarioScore
Per-scenario result.
Scorecard
The full scorecard.

Functions§

run_scorecard
Run the full scenario matrix and assemble the scorecard.