Expand description
Prediction calibration — Brier scorecard with the Murphy decomposition.
Tracks recorded predictions, resolves them against reality, and produces an honest calibration scorecard:
- Brier score: mean squared error
(p − o)²over resolved forecasts - Reliability: how well predicted probabilities match observed rates
- Resolution: how much predictions separate positive from negative outcomes
- Uncertainty: base rate
ō(1 − ō)— the difficulty of the problem - Brier skill score (BSS):
1 − Brier/Uncertaintyvs. climatology
The calibration gap feeds a small adjustment back into future predictions,
matching the v26 simulation.calibrate bridge but with the full
decomposition the v26 version lacked.
Structs§
- Brier
Scorecard - Full Brier scorecard with the Murphy decomposition.
- Calibration
Bin - A single calibration bin: predicted-probability range vs. observed rate.
- Calibration
Prediction - A single recorded prediction, resolved or pending.
- Calibration
Store - In-memory calibration store — shared per MCP server instance.