Module calibrate 

Recalibrate the serving threshold from real deferred feedback (SPEC §10.1, run against live traffic instead of a static benchmark suite) — the “learns your quality bar” loop.

Two calibration methods are available:

• conformal (default): split-conformal with Hoeffding bound — calibrate_from_store.
• ltt: Learn-then-Test / RCPS with exact-binomial fixed-sequence testing — calibrate_from_store_ltt.

Both enumerate stored traces, pair each trace that has a deferred outcome with the score of the attempt actually served, and hand the pairs to the respective core module. Neither feeds back into the request hot path — that wiring is a deliberate follow-on once an operator has reviewed a report.

Structs

CalibrationReport

The result of calibrating a conformal threshold against real deferred feedback.

LttReport

The result of LTT calibration against real deferred feedback.

Functions

calibrate_from_store

Calibrate a conformal threshold from every trace in the store that has a deferred outcome recorded.

calibrate_from_store_ltt

Calibrate an LTT threshold from every trace in the store that has a deferred outcome.

calibrate_pairs

Calibrate a conformal threshold from (score, correct) pairs — a thin wrapper over firstpass_core::conformal that also reports the empirical served-failure at the chosen threshold.

calibrate_pairs_ltt

Calibrate an LTT threshold from (score, correct) pairs — thin wrapper over firstpass_core::ltt.