Module calibration 

Plan J t-015: confidence-bucket calibration helpers.

Calibration asks: when our predictor (uncertainty.level, recovery estimate, result_bucket) says “medium”, how often is the underlying result actually correct?

A well-calibrated predictor’s pass rate within a bucket should roughly match the bucket’s nominal confidence. If medium has a 95% pass rate, it should probably be relabeled high. If high has a 60% pass rate, the threshold is too generous.

This module is pure (no I/O, no async, no clock). It takes a flat list of (predicted_bucket, passed) observations and computes per-bucket statistics. The probe harness collects the observations and feeds them in; the kernel here is the same regardless of where observations come from (probe-harness results, golden-set scoring, A/B test outcomes).

Structs

BucketStats

Per-bucket calibration statistics.

CalibrationReport

Aggregate calibration across all buckets.

Enums

AdviceStatus

Why a bucket’s advice field has the value it does.

Functions

compute_calibration

Compute calibration from a stream of (bucket_label, passed) observations. Buckets are grouped by exact-string label match (case-sensitive — caller should normalize first if needed).