Module conformal 

Split-conformal risk control on the gate threshold (SPEC §10.1).

The standing critique of every cascade is that its deferral threshold is a hand-tuned hyperparameter with no guarantee. We replace that with a calibrated one: given a held-out set of (gate_score, was_correct) pairs, choose the lowest score threshold λ such that serving everything scoring ≥ λ has a failure rate whose finite-sample upper confidence bound is ≤ α. Serving on score ≥ λ then carries a distribution-free guarantee: served-failure rate ≤ α at confidence 1 − δ. The bound is Hoeffding’s, so it is conservative — it never under-covers.

Structs

AdaptiveConformal

Online / adaptive conformal — Gibbs & Candès (2021), Adaptive Conformal Inference Under Distribution Shift. calibrate fixes a threshold ONCE and assumes exchangeability; under real drift (models change, prompts change, the gate’s error rate moves) the realized served-failure wanders off target. AdaptiveConformal instead tracks the serving threshold online from realized outcomes, so the long-run served-failure rate stays at alpha as the workload shifts.

ConformalResult

The result of calibrating a conformal threshold.

Functions

calibrate

Calibrate a conformal serving threshold.

served_failure_rate

Empirical served-failure rate on a fresh set at a given threshold (for validation).