Skip to main content

firstpass_core/
eprocess.rs

1//! Anytime-valid risk control on the gate threshold — a bound that holds at **every round**
2//! (SPEC §10.1, the online regime).
3//!
4//! ## Why this exists when [`crate::conformal`] and [`crate::ltt`] already do risk control
5//!
6//! The three regimes already shipped are each valid for what they claim, and none of them covers
7//! the way an operator actually runs the proxy:
8//!
9//! | existing | guarantee | why it is not enough here |
10//! |---|---|---|
11//! | [`crate::conformal::calibrate`] | served-failure ≤ `alpha` at confidence `1 − delta` | assumes **exchangeability**, and fixes `λ` once |
12//! | [`crate::ltt`] | FWER-controlled, exact-binomial, no exchangeability needed | **fixed-sample**: one calibration set, one decision |
13//! | [`crate::conformal::AdaptiveConformal`] | Gibbs–Candès ACI | keeps only the **long-run average** at `alpha` |
14//!
15//! Two concrete holes follow, and both bite in production:
16//!
17//! 1. **A long-run average is not a promise about now.** A process can sit above `alpha` for an
18//!    arbitrarily long stretch and still converge, because the excess is amortised over a horizon
19//!    the operator may never reach. Someone holding an error budget needs the bound to hold at the
20//!    round they are on.
21//! 2. **Repeated recalibration breaks fixed-sample guarantees.** LTT controls the family-wise error
22//!    rate over *one* pre-specified test sequence on *one* calibration set. Re-run it whenever more
23//!    feedback lands and adopt the winning threshold each time — which is exactly what a live
24//!    recalibration loop does — and that is optional stopping on a growing stream. The guarantee
25//!    does not compose across the repeats, and the realized risk can exceed `alpha` without any
26//!    single run of LTT ever being wrong.
27//!
28//! ## Method
29//!
30//! Khosravi & Huo (2026), *Conformal Selective Acting: Anytime-Valid Risk Control for RLVR-Trained
31//! LLMs* ([arXiv:2605.20270](https://arxiv.org/abs/2605.20270)).
32//!
33//! Maintain one **e-process** per candidate threshold on a fixed grid. An e-process is a
34//! non-negative supermartingale under the null, so **Ville's inequality** bounds it at *all*
35//! stopping times simultaneously:
36//!
37//! ```text
38//!     P( ∃t : E_t ≥ 1/delta )  ≤  delta        under H₀: risk(λ) > alpha
39//! ```
40//!
41//! That single quantifier — `∃t`, rather than a fixed `t` — is the whole point, and it is what
42//! makes the bound survive optional stopping. A threshold is **certified** once its e-value crosses
43//! `1/delta`, and certification is permanent: it was earned against every round up to that point,
44//! so later data cannot retroactively invalidate it (it can, and does, certify *more* thresholds).
45//!
46//! Per round, for a served item with gate score `s` and observed correctness `c`, each threshold
47//! `λ ≤ s` (i.e. each threshold that *would have served* this item) is updated by the betting factor
48//!
49//! ```text
50//!     E_λ ← E_λ · (1 + lambda_bet · (alpha − err))      where err = 1 if incorrect else 0
51//! ```
52//!
53//! Under H₀ the expected multiplier is `≤ 1`, which is precisely the supermartingale property.
54//! Thresholds that never serve are never updated, so an unused threshold neither gains nor loses
55//! evidence — a nuance the fixed-sample methods get for free but an online one has to be careful
56//! about.
57//!
58//! The grid is a **Bonferroni** family: with `n` candidate thresholds each tested at `delta / n`,
59//! the probability that *any* threshold is falsely certified at *any* round stays `≤ delta`. The
60//! served threshold is the **least conservative certified** one — the smallest `λ` whose e-process
61//! has crossed — which serves as much traffic as the evidence allows.
62//!
63//! ## The honest cost
64//!
65//! Anytime validity is not free. Against [`crate::conformal::AdaptiveConformal`] at the same
66//! `alpha` this serves **less** traffic, because it will not lower the threshold until the evidence
67//! has actually accumulated, and the Bonferroni correction makes each individual threshold harder
68//! to certify. That is the trade being offered: ACI tracks a shifting target aggressively and
69//! promises an average; this one promises every round and pays for it in conservatism. An operator
70//! picks the regime; neither is strictly better, and the ADR says so.
71//!
72//! Before any threshold is certified there is no guarantee to serve on, so
73//! [`EProcessRiskControl::certified_threshold`] returns `None` and the caller keeps its existing
74//! behaviour. Failing closed is the only safe reading of "I have not proven anything yet".
75
76use serde::{Deserialize, Serialize};
77
78/// Outcome of a certification query: the served threshold and the evidence behind it.
79#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
80pub struct Certification {
81    /// The least conservative certified threshold — serve iff `score >= threshold`.
82    pub threshold: f64,
83    /// The e-value backing it. Always `>= 1/delta` for a certified threshold.
84    pub e_value: f64,
85    /// Rounds observed when this certification was first reached.
86    pub certified_at_round: u64,
87}
88
89/// Default betting fraction. Conservative on purpose: a large bet grows the e-process fast when the
90/// null is false but decays it just as fast when it is true, and a decayed e-process takes a long
91/// time to recover. `0.5` is the standard "half-Kelly" compromise.
92pub const DEFAULT_BET: f64 = 0.5;
93
94/// Default e-value ceiling, as a multiple of the crossing level.
95///
96/// Truncating a supermartingale from above leaves it a supermartingale, so the type-I guarantee is
97/// unaffected — this is purely about **responsiveness**. Without a cap, a long clean stretch banks
98/// unbounded surplus evidence, and a subsequent regime change must burn all of it off before the
99/// threshold de-certifies; the bound holds asymptotically while realized failure sits above `alpha`
100/// for thousands of rounds. Capping bounds that de-certification lag. `10.0` keeps roughly an order
101/// of magnitude of headroom — enough that ordinary noise does not flap the certification, small
102/// enough that a genuine shift is caught in tens of rounds rather than thousands.
103pub const DEFAULT_CAP: f64 = 10.0;
104
105/// Anytime-valid risk controller over a fixed grid of candidate thresholds.
106///
107/// Feed it the deferred-feedback stream via [`EProcessRiskControl::observe_served`] and read
108/// [`EProcessRiskControl::certified_threshold`] on the router hot path. Unlike
109/// [`crate::conformal::AdaptiveConformal`], the answer is valid at the round you read it.
110#[derive(Debug, Clone, Serialize, Deserialize)]
111pub struct EProcessRiskControl {
112    /// Target served-failure rate.
113    alpha: f64,
114    /// Family-wise error budget across the whole grid and all rounds.
115    delta: f64,
116    /// Betting fraction in `(0, 1]`.
117    bet: f64,
118    /// E-values are capped at `cap * crossing_level`, bounding how much surplus evidence a clean
119    /// stretch can bank against a later regime change. See [`DEFAULT_CAP`].
120    cap: f64,
121    /// Candidate thresholds, ascending.
122    grid: Vec<f64>,
123    /// Running e-value per grid point, parallel to `grid`. Starts at `1.0` (no evidence).
124    e_values: Vec<f64>,
125    /// Round at which each grid point first crossed, if it has.
126    certified_at: Vec<Option<u64>>,
127    /// Rounds observed.
128    rounds: u64,
129    /// Served count and failures, for diagnostics only — never part of the guarantee.
130    served: u64,
131    served_fails: u64,
132}
133
134impl EProcessRiskControl {
135    /// Build a controller over `grid` candidate thresholds.
136    ///
137    /// `alpha` is the target served-failure rate, `delta` the family-wise error budget (the bound
138    /// holds with probability `1 − delta` across every threshold and every round jointly).
139    ///
140    /// The grid is sorted and de-duplicated; an empty grid yields a controller that never certifies,
141    /// which is the correct degenerate behaviour rather than an error — the caller then simply keeps
142    /// its existing threshold.
143    #[must_use]
144    pub fn new(alpha: f64, delta: f64, bet: f64, grid: &[f64]) -> Self {
145        Self::with_cap(alpha, delta, bet, DEFAULT_CAP, grid)
146    }
147
148    /// As [`EProcessRiskControl::new`], with an explicit e-value ceiling. See [`DEFAULT_CAP`] for
149    /// why a ceiling exists and what it trades.
150    #[must_use]
151    pub fn with_cap(alpha: f64, delta: f64, bet: f64, cap: f64, grid: &[f64]) -> Self {
152        // The betting factor on a failure is `1 + bet*(alpha - 1)`, which reaches zero at
153        // `bet = 1/(1 - alpha)` and goes negative beyond it. A zero factor annihilates the e-value
154        // permanently: no amount of subsequent evidence can revive a product that has hit exactly
155        // 0, so a single failure would silently retire that threshold forever. Clamping the bet
156        // just below the annihilating value keeps the process a strict supermartingale (which is
157        // what Ville's inequality needs) while leaving it able to recover.
158        //
159        // Flagged in review. Reachable through the public `with_cap` and by config, so it is a
160        // real input to validate rather than a theoretical corner.
161        // Bound the failure factor at 0.5 rather than merely above 0. Clamping at `0.99/(1-alpha)`
162        // technically avoids annihilation but leaves a factor of 0.01 — a single failure costs 100x
163        // and takes ~50 consecutive successes to undo, which is dead in practice if not in theory.
164        // The first version of this clamp did exactly that, and the recovery half of the test below
165        // failed: the controller never certified again. Half per failure is recoverable and still
166        // strictly decreasing, which is all the supermartingale property needs.
167        let max_bet = 0.5 / (1.0 - alpha).max(f64::EPSILON);
168        let bet = if bet.is_finite() {
169            bet.clamp(0.0, max_bet)
170        } else {
171            DEFAULT_BET.min(max_bet)
172        };
173        let mut grid: Vec<f64> = grid.iter().copied().filter(|g| g.is_finite()).collect();
174        grid.sort_by(|a, b| a.partial_cmp(b).unwrap_or(std::cmp::Ordering::Equal));
175        grid.dedup();
176        let n = grid.len();
177        Self {
178            alpha,
179            delta,
180            bet,
181            cap,
182            grid,
183            e_values: vec![1.0; n],
184            certified_at: vec![None; n],
185            rounds: 0,
186            served: 0,
187            served_fails: 0,
188        }
189    }
190
191    /// The per-threshold crossing level under Bonferroni: `n / delta`.
192    ///
193    /// Splitting `delta` across the `n` grid points is what keeps the *family-wise* error at
194    /// `delta`. Testing every threshold at the full `delta` would inflate the false-certification
195    /// rate with grid size — the classic multiple-testing trap, and an easy one to fall into here
196    /// because each individual e-process looks perfectly valid on its own.
197    #[must_use]
198    pub fn crossing_level(&self) -> f64 {
199        if self.grid.is_empty() || self.delta <= 0.0 {
200            return f64::INFINITY;
201        }
202        self.grid.len() as f64 / self.delta
203    }
204
205    /// Record one served item: its gate `score`, and whether it turned out correct.
206    ///
207    /// Only thresholds that *would have served* this item (`λ <= score`) are updated — a threshold
208    /// strict enough to have escalated instead has learned nothing about its own risk from it.
209    pub fn observe_served(&mut self, score: f64, was_correct: bool) {
210        self.rounds += 1;
211        self.served += 1;
212        self.served_fails += u64::from(!was_correct);
213
214        let err = f64::from(!was_correct);
215        let level = self.crossing_level();
216
217        for i in 0..self.grid.len() {
218            if self.grid[i] > score {
219                continue;
220            }
221            // Betting factor. Under H₀ (true risk > alpha) the expectation is <= 1, which is the
222            // supermartingale property Ville's inequality needs. Clamped at zero because a negative
223            // e-value is meaningless and would flip the comparison below.
224            let factor = (1.0 + self.bet * (self.alpha - err)).max(0.0);
225            // Capped. Ville's inequality is indifferent to the cap (truncating a supermartingale
226            // from above leaves it a supermartingale, so type-I control is untouched), but the
227            // operational difference is the whole ballgame: an uncapped e-value compounds through a
228            // long clean stretch to something astronomically above the level, and then a regime
229            // change has to burn through all of it before the threshold de-certifies. The bound
230            // would still hold in the limit while the realized rate sat above alpha for thousands
231            // of rounds — which is precisely the "long-run average" failure this module rejects.
232            self.e_values[i] = (self.e_values[i] * factor).min(self.cap * level);
233            if self.certified_at[i].is_none() && self.e_values[i] >= level {
234                self.certified_at[i] = Some(self.rounds);
235            }
236        }
237    }
238
239    /// The least conservative certified threshold, or `None` if nothing is certified yet.
240    ///
241    /// `None` means "no threshold has earned a guarantee", and the caller must keep its existing
242    /// behaviour rather than invent one. Serving on an uncertified threshold would be exactly the
243    /// unproven-claim failure this module exists to prevent.
244    /// Certification is evaluated **live**, against the current e-value — it is not a latch.
245    ///
246    /// This is the subtle part, and getting it wrong silently reintroduces the very failure mode the
247    /// module exists to prevent. A threshold that crossed during a clean stretch and has since been
248    /// contradicted by evidence is *not* certified any more, and must stop serving. Latching it —
249    /// "it was proven once, so it stays proven" — is what a fixed-sample method does, and under a
250    /// regime change it keeps serving at a threshold the current regime does not justify. The first
251    /// draft of this module latched, and `coverage_holds_at_every_round_under_a_regime_change`
252    /// caught it: realized failure hit 0.312 against alpha=0.20.
253    ///
254    /// `certified_at_round` is retained as provenance — when the evidence *first* crossed — which is
255    /// useful in a receipt and harmless here, because the serving decision reads the e-value.
256    #[must_use]
257    pub fn certified_threshold(&self) -> Option<Certification> {
258        let level = self.crossing_level();
259        self.grid
260            .iter()
261            .enumerate()
262            .find(|(i, _)| self.e_values[*i] >= level)
263            .map(|(i, &threshold)| Certification {
264                threshold,
265                e_value: self.e_values[i],
266                certified_at_round: self.certified_at[i].unwrap_or(self.rounds),
267            })
268    }
269
270    /// Whether `score` clears the certified threshold. `false` while nothing is certified.
271    #[must_use]
272    pub fn should_serve(&self, score: f64) -> bool {
273        self.certified_threshold()
274            .is_some_and(|c| score >= c.threshold)
275    }
276
277    /// Realized served-failure rate so far — a running diagnostic, not the guarantee.
278    #[must_use]
279    pub fn realized_served_failure(&self) -> f64 {
280        if self.served == 0 {
281            0.0
282        } else {
283            self.served_fails as f64 / self.served as f64
284        }
285    }
286
287    /// Rounds observed.
288    #[must_use]
289    pub const fn rounds(&self) -> u64 {
290        self.rounds
291    }
292
293    /// Current e-value at each grid point, parallel to the grid — for telemetry and tests.
294    #[must_use]
295    pub fn e_values(&self) -> &[f64] {
296        &self.e_values
297    }
298
299    /// The candidate grid, ascending.
300    #[must_use]
301    pub fn grid(&self) -> &[f64] {
302        &self.grid
303    }
304}
305
306#[cfg(test)]
307mod tests {
308    use super::*;
309
310    /// Deterministic uniform stream in `[0,1)`. A seeded LCG rather than `rand`, so the evidence
311    /// these tests produce is reproducible by an auditor on any machine, which is the same reason
312    /// the rest of the crate avoids ambient randomness.
313    struct Lcg(u64);
314    impl Lcg {
315        fn next_f64(&mut self) -> f64 {
316            self.0 = self
317                .0
318                .wrapping_mul(6_364_136_223_846_793_005)
319                .wrapping_add(1);
320            ((self.0 >> 11) as f64) / ((1u64 << 53) as f64)
321        }
322    }
323
324    fn grid() -> Vec<f64> {
325        (0..=20).map(|i| f64::from(i) / 20.0).collect()
326    }
327
328    /// A well-behaved gate: high score => usually correct.
329    fn correct_for(score: f64, u: f64) -> bool {
330        u < score
331    }
332
333    /// Nothing may be served before evidence exists. This is the fail-closed property: an
334    /// uncertified controller must not hand out a threshold, or the guarantee is decorative.
335    #[test]
336    fn nothing_is_certified_before_any_evidence() {
337        let e = EProcessRiskControl::new(0.1, 0.05, DEFAULT_BET, &grid());
338        assert!(
339            e.certified_threshold().is_none(),
340            "a fresh controller must certify nothing"
341        );
342        assert!(
343            !e.should_serve(1.0),
344            "even a perfect score must not serve without a certified threshold"
345        );
346    }
347
348    /// On a clean stream the controller must eventually certify — a method that never certifies is
349    /// trivially "safe" and completely useless, so this is the liveness half of the contract.
350    #[test]
351    fn a_clean_stream_eventually_certifies() {
352        let mut e = EProcessRiskControl::new(0.2, 0.05, DEFAULT_BET, &grid());
353        let mut rng = Lcg(42);
354        for _ in 0..4000 {
355            let score = 0.90 + rng.next_f64() * 0.10;
356            let correct = correct_for(score, rng.next_f64());
357            e.observe_served(score, correct);
358        }
359        let cert = e
360            .certified_threshold()
361            .expect("4000 clean rounds must certify some threshold");
362        assert!(
363            cert.e_value >= e.crossing_level(),
364            "a certified threshold must sit at or above the Bonferroni crossing level: {} < {}",
365            cert.e_value,
366            e.crossing_level()
367        );
368    }
369
370    /// **The core guarantee, and the reason this module exists.**
371    ///
372    /// Anytime-validity means the bound holds at EVERY round, not just at the end. This checks the
373    /// realized served-failure rate at every single round after certification, on an ADVERSARIAL
374    /// non-exchangeable stream: a clean first half, then an abrupt regime change where the gate's
375    /// error rate jumps. Split conformal's exchangeability assumption is violated by construction
376    /// here — see `split_conformal_breaches_where_the_e_process_holds` for the contrast.
377    #[test]
378    fn coverage_holds_at_every_round_under_a_regime_change() {
379        let alpha = 0.20;
380        let mut worst: f64 = 0.0;
381        for seed in 0..40u64 {
382            let mut e = EProcessRiskControl::new(alpha, 0.05, DEFAULT_BET, &grid());
383            let mut rng = Lcg(seed.wrapping_mul(2_654_435_761).wrapping_add(7));
384            let mut served = 0u64;
385            let mut fails = 0u64;
386            for t in 0..3000 {
387                let score = 0.80 + rng.next_f64() * 0.20;
388                // Regime change at the midpoint: the gate becomes much less reliable, with no
389                // warning and no return. This is exactly what exchangeability forbids.
390                let correct = if t < 1500 {
391                    correct_for(score, rng.next_f64())
392                } else {
393                    rng.next_f64() < 0.55
394                };
395                if e.should_serve(score) {
396                    served += 1;
397                    fails += u64::from(!correct);
398                    // Check the realized rate at THIS round, not only at the end.
399                    if served >= 50 {
400                        let rate = fails as f64 / served as f64;
401                        worst = worst.max(rate);
402                    }
403                }
404                e.observe_served(score, correct);
405            }
406        }
407        // Ville's inequality is a high-probability bound, not an almost-sure one, so a small
408        // overshoot budget is correct rather than a fudge: at delta = 0.05 some excursion is
409        // permitted by the theory itself.
410        assert!(
411            worst <= alpha + 0.10,
412            "worst per-round served-failure {worst:.3} exceeded alpha={alpha} plus slack"
413        );
414    }
415
416    /// **The contrast that makes the contribution.**
417    ///
418    /// On the same non-exchangeable stream, a split-conformal threshold fitted on the first regime
419    /// carries its guarantee only under exchangeability — which the regime change destroys. It goes
420    /// on serving at a threshold the new regime does not justify, and its realized failure rate
421    /// breaches alpha. The e-process, having to re-earn its evidence every round, does not.
422    ///
423    /// If this test ever fails it means the streams stopped being adversarial, not that the method
424    /// improved — so it asserts the breach explicitly rather than just comparing the two.
425    #[test]
426    fn split_conformal_breaches_where_the_e_process_holds() {
427        let alpha = 0.15;
428        let n = 6000;
429        let mut rng = Lcg(20_260_814);
430
431        // Phase 1 doubles as split-conformal's calibration set.
432        let mut calib: Vec<(f64, bool)> = Vec::new();
433        for _ in 0..1000 {
434            let score = 0.70 + rng.next_f64() * 0.30;
435            calib.push((score, correct_for(score, rng.next_f64())));
436        }
437        let fixed = crate::conformal::calibrate(&calib, alpha, 0.05, 30).threshold;
438
439        let mut e = EProcessRiskControl::new(alpha, 0.05, DEFAULT_BET, &grid());
440        // Warm the e-process on the same phase-1 data, so neither method gets a data advantage.
441        for &(s, c) in &calib {
442            e.observe_served(s, c);
443        }
444
445        let (mut sc_served, mut sc_fails) = (0u64, 0u64);
446        let (mut ep_served, mut ep_fails) = (0u64, 0u64);
447        for t in 0..n {
448            let score = 0.70 + rng.next_f64() * 0.30;
449            // Phase 2: the relationship between score and correctness collapses.
450            let correct = if t < n / 2 {
451                correct_for(score, rng.next_f64())
452            } else {
453                rng.next_f64() < 0.60
454            };
455            if score >= fixed {
456                sc_served += 1;
457                sc_fails += u64::from(!correct);
458            }
459            if e.should_serve(score) {
460                ep_served += 1;
461                ep_fails += u64::from(!correct);
462            }
463            e.observe_served(score, correct);
464        }
465
466        let sc_rate = sc_fails as f64 / sc_served.max(1) as f64;
467        let ep_rate = ep_fails as f64 / ep_served.max(1) as f64;
468        assert!(
469            sc_rate > alpha,
470            "the stream is meant to break split conformal, but its rate was {sc_rate:.3} <= alpha={alpha} \
471             — the test has stopped being adversarial and proves nothing"
472        );
473        assert!(
474            ep_rate < sc_rate,
475            "the e-process ({ep_rate:.3}) must beat split conformal ({sc_rate:.3}) under shift"
476        );
477    }
478
479    /// **The cap must bound how long a stale certification survives.**
480    ///
481    /// Written because the first mutation run exposed that nothing covered the cap: deleting it
482    /// left all other tests green. Without a ceiling, a long clean stretch banks unbounded surplus
483    /// evidence, and after the regime turns bad the threshold keeps serving until that surplus is
484    /// burned off — the bound still holds in the limit while realized failure sits above `alpha`
485    /// for thousands of rounds. That is the long-run-average failure mode wearing a new hat.
486    ///
487    /// Capped vs uncapped on an identical stream, measuring rounds-to-de-certify.
488    #[test]
489    fn the_cap_bounds_how_long_a_stale_certification_survives() {
490        let alpha = 0.10;
491        let lag = |cap: f64| -> u64 {
492            let mut e = EProcessRiskControl::with_cap(alpha, 0.05, DEFAULT_BET, cap, &grid());
493            let mut rng = Lcg(99);
494            // Long clean stretch: bank evidence.
495            for _ in 0..5000 {
496                let score = 0.95 + rng.next_f64() * 0.05;
497                e.observe_served(score, correct_for(score, rng.next_f64()));
498            }
499            assert!(
500                e.certified_threshold().is_some(),
501                "the clean stretch must certify, or the lag measurement is meaningless"
502            );
503            // Regime turns hostile: every served item now fails. Count rounds to de-certify.
504            let mut rounds = 0u64;
505            while e.certified_threshold().is_some() && rounds < 100_000 {
506                e.observe_served(0.99, false);
507                rounds += 1;
508            }
509            rounds
510        };
511
512        let capped = lag(DEFAULT_CAP);
513        let uncapped = lag(f64::INFINITY);
514        assert!(
515            capped < uncapped,
516            "the cap must shorten de-certification lag: capped={capped} uncapped={uncapped}"
517        );
518        assert!(
519            capped <= 100,
520            "a capped controller must drop a contradicted certification within ~tens of rounds, took {capped}"
521        );
522    }
523
524    /// **A large bet must not permanently annihilate a threshold.**
525    ///
526    /// Flagged in review. The failure factor is `1 + bet*(alpha - 1)`, which hits exactly zero at
527    /// `bet = 1/(1-alpha)` and goes negative past it. Zero is absorbing under multiplication: one
528    /// failure would retire that threshold forever, and no later evidence could revive it. The
529    /// controller would look like it was still learning while being permanently dead.
530    ///
531    /// `bet` is reachable through the public `with_cap` and through config, so it is validated
532    /// input, not a theoretical corner.
533    #[test]
534    fn an_oversized_bet_cannot_permanently_kill_a_threshold() {
535        let alpha = 0.20;
536        // bet = 5.0 is far past the annihilating 1/(1-0.2) = 1.25.
537        let mut e = EProcessRiskControl::with_cap(alpha, 0.05, 5.0, DEFAULT_CAP, &grid());
538        e.observe_served(0.9, false);
539        assert!(
540            e.e_values().iter().all(|v| *v > 0.0),
541            "a failure must shrink e-values, never zero them: {:?}",
542            e.e_values()
543        );
544
545        // And the process must still be able to recover and certify afterwards.
546        let mut rng = Lcg(7);
547        for _ in 0..4000 {
548            let score = 0.95 + rng.next_f64() * 0.05;
549            e.observe_served(score, correct_for(score, rng.next_f64()));
550        }
551        assert!(
552            e.certified_threshold().is_some(),
553            "a controller that took one early failure must still be able to certify later"
554        );
555    }
556
557    /// **Precedence contract with ACI**, verified at the source rather than trusted in prose.
558    ///
559    /// `proxy.rs` prefers the e-process threshold over `AdaptiveConformal`'s whenever one is
560    /// certified, and falls through otherwise. That is only safe because an uncertified controller
561    /// returns `None` rather than a default — if it ever returned a number before earning it, the
562    /// router would silently serve on an unproven threshold and outrank a method that at least has
563    /// a long-run guarantee.
564    #[test]
565    fn an_uncertified_controller_yields_nothing_for_the_router_to_prefer() {
566        let mut e = EProcessRiskControl::new(0.2, 0.05, DEFAULT_BET, &grid());
567        // Some evidence, but nowhere near the crossing level.
568        for _ in 0..10 {
569            e.observe_served(0.95, true);
570        }
571        assert!(
572            e.certified_threshold().is_none(),
573            "ten rounds must not certify at delta=0.05 over a 21-point grid"
574        );
575        // ...and once it HAS earned it, the router gets a real threshold to prefer.
576        let mut rng = Lcg(11);
577        for _ in 0..4000 {
578            let score = 0.95 + rng.next_f64() * 0.05;
579            e.observe_served(score, correct_for(score, rng.next_f64()));
580        }
581        let c = e
582            .certified_threshold()
583            .expect("sustained clean evidence must certify");
584        assert!(
585            (0.0..=1.0).contains(&c.threshold),
586            "a certified threshold must be servable: {}",
587            c.threshold
588        );
589    }
590
591    /// `delta = 0` means "zero error budget", and must certify NOTHING rather than everything.
592    ///
593    /// Review raised this as a possible false-certification path. It is already handled —
594    /// `crossing_level` returns infinity for a non-positive delta — but "already correct" is not a
595    /// reason to leave a fail-closed contract untested, and an infinity comparison is exactly the
596    /// kind of thing a later refactor breaks silently.
597    #[test]
598    fn a_zero_error_budget_certifies_nothing() {
599        for delta in [0.0, -0.1] {
600            let mut e = EProcessRiskControl::new(0.2, delta, DEFAULT_BET, &grid());
601            let mut rng = Lcg(3);
602            for _ in 0..5000 {
603                let score = 0.99;
604                e.observe_served(score, correct_for(score, rng.next_f64()));
605            }
606            assert!(
607                e.crossing_level().is_infinite(),
608                "delta={delta} must make the crossing level unreachable"
609            );
610            assert!(
611                e.certified_threshold().is_none(),
612                "delta={delta} is a zero error budget: nothing may ever be certified"
613            );
614            assert!(!e.should_serve(1.0), "and nothing may be served");
615        }
616    }
617
618    /// Bonferroni: the crossing level must scale with grid size. Testing every threshold at the
619    /// full `delta` is the multiple-testing trap — each e-process looks valid alone, while the
620    /// family-wise false-certification rate quietly grows with the grid.
621    #[test]
622    fn the_crossing_level_scales_with_grid_size() {
623        let small = EProcessRiskControl::new(0.1, 0.05, DEFAULT_BET, &[0.5, 0.6]);
624        let large = EProcessRiskControl::new(0.1, 0.05, DEFAULT_BET, &grid());
625        assert!(
626            large.crossing_level() > small.crossing_level(),
627            "a larger grid must demand more evidence per threshold: {} vs {}",
628            large.crossing_level(),
629            small.crossing_level()
630        );
631        assert!(
632            (small.crossing_level() - 2.0 / 0.05).abs() < 1e-9,
633            "crossing level must be exactly n/delta"
634        );
635    }
636
637    /// A threshold strict enough to have escalated an item learns nothing from it. Without this,
638    /// unused thresholds would accumulate evidence they never earned and certify on traffic they
639    /// would never have served.
640    #[test]
641    fn thresholds_that_would_not_have_served_are_not_updated() {
642        let mut e = EProcessRiskControl::new(0.1, 0.05, DEFAULT_BET, &[0.2, 0.9]);
643        e.observe_served(0.5, true);
644        let vals = e.e_values();
645        assert!(
646            (vals[1] - 1.0).abs() < f64::EPSILON,
647            "threshold 0.9 did not serve a 0.5-scoring item, so its e-value must stay 1.0, got {}",
648            vals[1]
649        );
650        assert!(
651            vals[0] > 1.0,
652            "threshold 0.2 did serve it correctly, so its e-value must grow, got {}",
653            vals[0]
654        );
655    }
656
657    /// A degenerate grid must be inert, not a panic. `new` takes a slice from config, and an empty
658    /// or non-finite grid is a plausible misconfiguration that must fail closed.
659    #[test]
660    fn a_degenerate_grid_certifies_nothing_and_does_not_panic() {
661        let mut e = EProcessRiskControl::new(0.1, 0.05, DEFAULT_BET, &[f64::NAN, f64::INFINITY]);
662        e.observe_served(0.9, true);
663        assert!(e.certified_threshold().is_none());
664        assert!(!e.should_serve(1.0));
665        assert!(
666            e.crossing_level().is_infinite(),
667            "an empty grid can never cross"
668        );
669    }
670}