Skip to main content

Module calibration

Module calibration 

Source
Expand description

Prediction calibration — Brier scorecard with the Murphy decomposition.

Tracks recorded predictions, resolves them against reality, and produces an honest calibration scorecard:

  • Brier score: mean squared error (p − o)² over resolved forecasts
  • Reliability: how well predicted probabilities match observed rates
  • Resolution: how much predictions separate positive from negative outcomes
  • Uncertainty: base rate ō(1 − ō) — the difficulty of the problem
  • Brier skill score (BSS): 1 − Brier/Uncertainty vs. climatology

The calibration gap feeds a small adjustment back into future predictions, matching the v26 simulation.calibrate bridge but with the full decomposition the v26 version lacked.

Structs§

BrierScorecard
Full Brier scorecard with the Murphy decomposition.
CalibrationBin
A single calibration bin: predicted-probability range vs. observed rate.
CalibrationPrediction
A single recorded prediction, resolved or pending.
CalibrationStore
In-memory calibration store — shared per MCP server instance.