Skip to main content

firstpass_core/
eprocess.rs

1//! Anytime-valid risk control on the gate threshold — a bound that holds at **every round**
2//! (SPEC §10.1, the online regime).
3//!
4//! ## Why this exists when [`crate::conformal`] and [`crate::ltt`] already do risk control
5//!
6//! The three regimes already shipped are each valid for what they claim, and none of them covers
7//! the way an operator actually runs the proxy:
8//!
9//! | existing | guarantee | why it is not enough here |
10//! |---|---|---|
11//! | [`crate::conformal::calibrate`] | served-failure ≤ `alpha` at confidence `1 − delta` | assumes **exchangeability**, and fixes `λ` once |
12//! | [`crate::ltt`] | FWER-controlled, exact-binomial, no exchangeability needed | **fixed-sample**: one calibration set, one decision |
13//! | [`crate::conformal::AdaptiveConformal`] | Gibbs–Candès ACI | keeps only the **long-run average** at `alpha` |
14//!
15//! Two concrete holes follow, and both bite in production:
16//!
17//! 1. **A long-run average is not a promise about now.** A process can sit above `alpha` for an
18//!    arbitrarily long stretch and still converge, because the excess is amortised over a horizon
19//!    the operator may never reach. Someone holding an error budget needs the bound to hold at the
20//!    round they are on.
21//! 2. **Repeated recalibration breaks fixed-sample guarantees.** LTT controls the family-wise error
22//!    rate over *one* pre-specified test sequence on *one* calibration set. Re-run it whenever more
23//!    feedback lands and adopt the winning threshold each time — which is exactly what a live
24//!    recalibration loop does — and that is optional stopping on a growing stream. The guarantee
25//!    does not compose across the repeats, and the realized risk can exceed `alpha` without any
26//!    single run of LTT ever being wrong.
27//!
28//! ## Method
29//!
30//! Khosravi & Huo (2026), *Conformal Selective Acting: Anytime-Valid Risk Control for RLVR-Trained
31//! LLMs* ([arXiv:2605.20270](https://arxiv.org/abs/2605.20270)).
32//!
33//! Maintain one **e-process** per candidate threshold on a fixed grid. An e-process is a
34//! non-negative supermartingale under the null, so **Ville's inequality** bounds it at *all*
35//! stopping times simultaneously:
36//!
37//! ```text
38//!     P( ∃t : E_t ≥ 1/delta )  ≤  delta        under H₀: risk(λ) > alpha
39//! ```
40//!
41//! That single quantifier — `∃t`, rather than a fixed `t` — is the whole point, and it is what
42//! makes the bound survive optional stopping. A threshold is **certified** once its e-value crosses
43//! `1/delta`, and certification is permanent: it was earned against every round up to that point,
44//! so later data cannot retroactively invalidate it (it can, and does, certify *more* thresholds).
45//!
46//! Per round, for a served item with gate score `s` and observed correctness `c`, each threshold
47//! `λ ≤ s` (i.e. each threshold that *would have served* this item) is updated by the betting factor
48//!
49//! ```text
50//!     E_λ ← E_λ · (1 + lambda_bet · (alpha − err))      where err = 1 if incorrect else 0
51//! ```
52//!
53//! Under H₀ the expected multiplier is `≤ 1`, which is precisely the supermartingale property.
54//! Thresholds that never serve are never updated, so an unused threshold neither gains nor loses
55//! evidence — a nuance the fixed-sample methods get for free but an online one has to be careful
56//! about.
57//!
58//! The grid is a **Bonferroni** family: with `n` candidate thresholds each tested at `delta / n`,
59//! the probability that *any* threshold is falsely certified at *any* round stays `≤ delta`. The
60//! served threshold is the **least conservative certified** one — the smallest `λ` whose e-process
61//! has crossed — which serves as much traffic as the evidence allows.
62//!
63//! ## The honest cost
64//!
65//! Anytime validity is not free. Against [`crate::conformal::AdaptiveConformal`] at the same
66//! `alpha` this serves **less** traffic, because it will not lower the threshold until the evidence
67//! has actually accumulated, and the Bonferroni correction makes each individual threshold harder
68//! to certify. That is the trade being offered: ACI tracks a shifting target aggressively and
69//! promises an average; this one promises every round and pays for it in conservatism. An operator
70//! picks the regime; neither is strictly better, and the ADR says so.
71//!
72//! Before any threshold is certified there is no guarantee to serve on, so
73//! [`EProcessRiskControl::certified_threshold`] returns `None` and the caller keeps its existing
74//! behaviour. Failing closed is the only safe reading of "I have not proven anything yet".
75
76use serde::{Deserialize, Serialize};
77
78/// Outcome of a certification query: the served threshold and the evidence behind it.
79#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
80pub struct Certification {
81    /// The least conservative certified threshold — serve iff `score >= threshold`.
82    pub threshold: f64,
83    /// The e-value backing it. Always `>= 1/delta` for a certified threshold.
84    pub e_value: f64,
85    /// Rounds observed when this certification was first reached.
86    pub certified_at_round: u64,
87}
88
89/// Default betting fraction. Conservative on purpose: a large bet grows the e-process fast when the
90/// null is false but decays it just as fast when it is true, and a decayed e-process takes a long
91/// time to recover. `0.5` is the standard "half-Kelly" compromise.
92pub const DEFAULT_BET: f64 = 0.5;
93
94/// Default e-value ceiling, as a multiple of the crossing level.
95///
96/// Truncating a supermartingale from above leaves it a supermartingale, so the type-I guarantee is
97/// unaffected — this is purely about **responsiveness**. Without a cap, a long clean stretch banks
98/// unbounded surplus evidence, and a subsequent regime change must burn all of it off before the
99/// threshold de-certifies; the bound holds asymptotically while realized failure sits above `alpha`
100/// for thousands of rounds. Capping bounds that de-certification lag. `10.0` keeps roughly an order
101/// of magnitude of headroom — enough that ordinary noise does not flap the certification, small
102/// enough that a genuine shift is caught in tens of rounds rather than thousands.
103pub const DEFAULT_CAP: f64 = 10.0;
104
105/// Anytime-valid risk controller over a fixed grid of candidate thresholds.
106///
107/// Feed it the deferred-feedback stream via [`EProcessRiskControl::observe_served`] and read
108/// [`EProcessRiskControl::certified_threshold`] on the router hot path. Unlike
109/// [`crate::conformal::AdaptiveConformal`], the answer is valid at the round you read it.
110#[derive(Debug, Clone, Serialize, Deserialize)]
111pub struct EProcessRiskControl {
112    /// Target served-failure rate.
113    alpha: f64,
114    /// Family-wise error budget across the whole grid and all rounds.
115    delta: f64,
116    /// Betting fraction in `(0, 1]`.
117    bet: f64,
118    /// E-values are capped at `cap * crossing_level`, bounding how much surplus evidence a clean
119    /// stretch can bank against a later regime change. See [`DEFAULT_CAP`].
120    cap: f64,
121    /// Candidate thresholds, ascending.
122    grid: Vec<f64>,
123    /// Running e-value per grid point, parallel to `grid`. Starts at `1.0` (no evidence).
124    e_values: Vec<f64>,
125    /// Round at which each grid point first crossed, if it has.
126    certified_at: Vec<Option<u64>>,
127    /// Rounds observed.
128    rounds: u64,
129    /// Served count and failures, for diagnostics only — never part of the guarantee.
130    served: u64,
131    served_fails: u64,
132}
133
134impl EProcessRiskControl {
135    /// Build a controller over `grid` candidate thresholds.
136    ///
137    /// `alpha` is the target served-failure rate, `delta` the family-wise error budget (the bound
138    /// holds with probability `1 − delta` across every threshold and every round jointly).
139    ///
140    /// The grid is sorted and de-duplicated; an empty grid yields a controller that never certifies,
141    /// which is the correct degenerate behaviour rather than an error — the caller then simply keeps
142    /// its existing threshold.
143    #[must_use]
144    pub fn new(alpha: f64, delta: f64, bet: f64, grid: &[f64]) -> Self {
145        Self::with_cap(alpha, delta, bet, DEFAULT_CAP, grid)
146    }
147
148    /// As [`EProcessRiskControl::new`], with an explicit e-value ceiling. See [`DEFAULT_CAP`] for
149    /// why a ceiling exists and what it trades.
150    #[must_use]
151    pub fn with_cap(alpha: f64, delta: f64, bet: f64, cap: f64, grid: &[f64]) -> Self {
152        // The betting factor on a failure is `1 + bet*(alpha - 1)`, which reaches zero at
153        // `bet = 1/(1 - alpha)` and goes negative beyond it. A zero factor annihilates the e-value
154        // permanently: no amount of subsequent evidence can revive a product that has hit exactly
155        // 0, so a single failure would silently retire that threshold forever. Clamping the bet
156        // just below the annihilating value keeps the process a strict supermartingale (which is
157        // what Ville's inequality needs) while leaving it able to recover.
158        //
159        // Flagged in review. Reachable through the public `with_cap` and by config, so it is a
160        // real input to validate rather than a theoretical corner.
161        // Bound the failure factor at 0.5 rather than merely above 0. Clamping at `0.99/(1-alpha)`
162        // technically avoids annihilation but leaves a factor of 0.01 — a single failure costs 100x
163        // and takes ~50 consecutive successes to undo, which is dead in practice if not in theory.
164        // The first version of this clamp did exactly that, and the recovery half of the test below
165        // failed: the controller never certified again. Half per failure is recoverable and still
166        // strictly decreasing, which is all the supermartingale property needs.
167        let max_bet = 0.5 / (1.0 - alpha).max(f64::EPSILON);
168        let bet = if bet.is_finite() {
169            bet.clamp(0.0, max_bet)
170        } else {
171            DEFAULT_BET.min(max_bet)
172        };
173        let mut grid: Vec<f64> = grid.iter().copied().filter(|g| g.is_finite()).collect();
174        grid.sort_by(|a, b| a.partial_cmp(b).unwrap_or(std::cmp::Ordering::Equal));
175        grid.dedup();
176        let n = grid.len();
177        Self {
178            alpha,
179            delta,
180            bet,
181            cap,
182            grid,
183            e_values: vec![1.0; n],
184            certified_at: vec![None; n],
185            rounds: 0,
186            served: 0,
187            served_fails: 0,
188        }
189    }
190
191    /// The per-threshold crossing level under Bonferroni: `n / delta`.
192    ///
193    /// Splitting `delta` across the `n` grid points is what keeps the *family-wise* error at
194    /// `delta`. Testing every threshold at the full `delta` would inflate the false-certification
195    /// rate with grid size — the classic multiple-testing trap, and an easy one to fall into here
196    /// because each individual e-process looks perfectly valid on its own.
197    #[must_use]
198    pub fn crossing_level(&self) -> f64 {
199        if self.grid.is_empty() || self.delta <= 0.0 {
200            return f64::INFINITY;
201        }
202        self.grid.len() as f64 / self.delta
203    }
204
205    /// Record one served item: its gate `score`, and whether it turned out correct.
206    ///
207    /// Only thresholds that *would have served* this item (`λ <= score`) are updated — a threshold
208    /// strict enough to have escalated instead has learned nothing about its own risk from it.
209    pub fn observe_served(&mut self, score: f64, was_correct: bool) {
210        self.rounds += 1;
211        self.served += 1;
212        self.served_fails += u64::from(!was_correct);
213
214        let err = f64::from(!was_correct);
215        let level = self.crossing_level();
216
217        for i in 0..self.grid.len() {
218            if self.grid[i] > score {
219                continue;
220            }
221            // Betting factor. Under H₀ (true risk > alpha) the expectation is <= 1, which is the
222            // supermartingale property Ville's inequality needs. Clamped at zero because a negative
223            // e-value is meaningless and would flip the comparison below.
224            let factor = (1.0 + self.bet * (self.alpha - err)).max(0.0);
225            // Capped. Ville's inequality is indifferent to the cap (truncating a supermartingale
226            // from above leaves it a supermartingale, so type-I control is untouched), but the
227            // operational difference is the whole ballgame: an uncapped e-value compounds through a
228            // long clean stretch to something astronomically above the level, and then a regime
229            // change has to burn through all of it before the threshold de-certifies. The bound
230            // would still hold in the limit while the realized rate sat above alpha for thousands
231            // of rounds — which is precisely the "long-run average" failure this module rejects.
232            self.e_values[i] = (self.e_values[i] * factor).min(self.cap * level);
233            if self.certified_at[i].is_none() && self.e_values[i] >= level {
234                self.certified_at[i] = Some(self.rounds);
235            }
236        }
237    }
238
239    /// The least conservative certified threshold, or `None` if nothing is certified yet.
240    ///
241    /// `None` means "no threshold has earned a guarantee", and the caller must keep its existing
242    /// behaviour rather than invent one. Serving on an uncertified threshold would be exactly the
243    /// unproven-claim failure this module exists to prevent.
244    /// Certification is evaluated **live**, against the current e-value — it is not a latch.
245    ///
246    /// This is the subtle part, and getting it wrong silently reintroduces the very failure mode the
247    /// module exists to prevent. A threshold that crossed during a clean stretch and has since been
248    /// contradicted by evidence is *not* certified any more, and must stop serving. Latching it —
249    /// "it was proven once, so it stays proven" — is what a fixed-sample method does, and under a
250    /// regime change it keeps serving at a threshold the current regime does not justify. The first
251    /// draft of this module latched, and `coverage_holds_at_every_round_under_a_regime_change`
252    /// caught it: realized failure hit 0.312 against alpha=0.20.
253    ///
254    /// `certified_at_round` is retained as provenance — when the evidence *first* crossed — which is
255    /// useful in a receipt and harmless here, because the serving decision reads the e-value.
256    #[must_use]
257    pub fn certified_threshold(&self) -> Option<Certification> {
258        let level = self.crossing_level();
259        self.grid
260            .iter()
261            .enumerate()
262            .find(|(i, _)| self.e_values[*i] >= level)
263            .map(|(i, &threshold)| Certification {
264                threshold,
265                e_value: self.e_values[i],
266                certified_at_round: self.certified_at[i].unwrap_or(self.rounds),
267            })
268    }
269
270    /// Whether `score` clears the certified threshold. `false` while nothing is certified.
271    #[must_use]
272    pub fn should_serve(&self, score: f64) -> bool {
273        self.certified_threshold()
274            .is_some_and(|c| score >= c.threshold)
275    }
276
277    /// Realized served-failure rate so far — a running diagnostic, not the guarantee.
278    #[must_use]
279    pub fn realized_served_failure(&self) -> f64 {
280        if self.served == 0 {
281            0.0
282        } else {
283            self.served_fails as f64 / self.served as f64
284        }
285    }
286
287    /// Rounds observed.
288    #[must_use]
289    pub const fn rounds(&self) -> u64 {
290        self.rounds
291    }
292
293    /// Current e-value at each grid point, parallel to the grid — for telemetry and tests.
294    #[must_use]
295    pub fn e_values(&self) -> &[f64] {
296        &self.e_values
297    }
298
299    /// The candidate grid, ascending.
300    #[must_use]
301    pub fn grid(&self) -> &[f64] {
302        &self.grid
303    }
304
305    /// Target served-failure rate the certificate is stated at.
306    #[must_use]
307    pub const fn alpha(&self) -> f64 {
308        self.alpha
309    }
310
311    /// Family-wise error budget the certificate is stated at.
312    #[must_use]
313    pub const fn delta(&self) -> f64 {
314        self.delta
315    }
316}
317
318#[cfg(test)]
319mod tests {
320    use super::*;
321
322    /// Deterministic uniform stream in `[0,1)`. A seeded LCG rather than `rand`, so the evidence
323    /// these tests produce is reproducible by an auditor on any machine, which is the same reason
324    /// the rest of the crate avoids ambient randomness.
325    struct Lcg(u64);
326    impl Lcg {
327        fn next_f64(&mut self) -> f64 {
328            self.0 = self
329                .0
330                .wrapping_mul(6_364_136_223_846_793_005)
331                .wrapping_add(1);
332            ((self.0 >> 11) as f64) / ((1u64 << 53) as f64)
333        }
334    }
335
336    fn grid() -> Vec<f64> {
337        (0..=20).map(|i| f64::from(i) / 20.0).collect()
338    }
339
340    /// A well-behaved gate: high score => usually correct.
341    fn correct_for(score: f64, u: f64) -> bool {
342        u < score
343    }
344
345    /// Nothing may be served before evidence exists. This is the fail-closed property: an
346    /// uncertified controller must not hand out a threshold, or the guarantee is decorative.
347    #[test]
348    fn nothing_is_certified_before_any_evidence() {
349        let e = EProcessRiskControl::new(0.1, 0.05, DEFAULT_BET, &grid());
350        assert!(
351            e.certified_threshold().is_none(),
352            "a fresh controller must certify nothing"
353        );
354        assert!(
355            !e.should_serve(1.0),
356            "even a perfect score must not serve without a certified threshold"
357        );
358    }
359
360    /// On a clean stream the controller must eventually certify — a method that never certifies is
361    /// trivially "safe" and completely useless, so this is the liveness half of the contract.
362    #[test]
363    fn a_clean_stream_eventually_certifies() {
364        let mut e = EProcessRiskControl::new(0.2, 0.05, DEFAULT_BET, &grid());
365        let mut rng = Lcg(42);
366        for _ in 0..4000 {
367            let score = 0.90 + rng.next_f64() * 0.10;
368            let correct = correct_for(score, rng.next_f64());
369            e.observe_served(score, correct);
370        }
371        let cert = e
372            .certified_threshold()
373            .expect("4000 clean rounds must certify some threshold");
374        assert!(
375            cert.e_value >= e.crossing_level(),
376            "a certified threshold must sit at or above the Bonferroni crossing level: {} < {}",
377            cert.e_value,
378            e.crossing_level()
379        );
380    }
381
382    /// **The core guarantee, and the reason this module exists.**
383    ///
384    /// Anytime-validity means the bound holds at EVERY round, not just at the end. This checks the
385    /// realized served-failure rate at every single round after certification, on an ADVERSARIAL
386    /// non-exchangeable stream: a clean first half, then an abrupt regime change where the gate's
387    /// error rate jumps. Split conformal's exchangeability assumption is violated by construction
388    /// here — see `split_conformal_breaches_where_the_e_process_holds` for the contrast.
389    #[test]
390    fn coverage_holds_at_every_round_under_a_regime_change() {
391        let alpha = 0.20;
392        let mut worst: f64 = 0.0;
393        for seed in 0..40u64 {
394            let mut e = EProcessRiskControl::new(alpha, 0.05, DEFAULT_BET, &grid());
395            let mut rng = Lcg(seed.wrapping_mul(2_654_435_761).wrapping_add(7));
396            let mut served = 0u64;
397            let mut fails = 0u64;
398            for t in 0..3000 {
399                let score = 0.80 + rng.next_f64() * 0.20;
400                // Regime change at the midpoint: the gate becomes much less reliable, with no
401                // warning and no return. This is exactly what exchangeability forbids.
402                let correct = if t < 1500 {
403                    correct_for(score, rng.next_f64())
404                } else {
405                    rng.next_f64() < 0.55
406                };
407                if e.should_serve(score) {
408                    served += 1;
409                    fails += u64::from(!correct);
410                    // Check the realized rate at THIS round, not only at the end.
411                    if served >= 50 {
412                        let rate = fails as f64 / served as f64;
413                        worst = worst.max(rate);
414                    }
415                }
416                e.observe_served(score, correct);
417            }
418        }
419        // Ville's inequality is a high-probability bound, not an almost-sure one, so a small
420        // overshoot budget is correct rather than a fudge: at delta = 0.05 some excursion is
421        // permitted by the theory itself.
422        assert!(
423            worst <= alpha + 0.10,
424            "worst per-round served-failure {worst:.3} exceeded alpha={alpha} plus slack"
425        );
426    }
427
428    /// **The contrast that makes the contribution.**
429    ///
430    /// On the same non-exchangeable stream, a split-conformal threshold fitted on the first regime
431    /// carries its guarantee only under exchangeability — which the regime change destroys. It goes
432    /// on serving at a threshold the new regime does not justify, and its realized failure rate
433    /// breaches alpha. The e-process, having to re-earn its evidence every round, does not.
434    ///
435    /// If this test ever fails it means the streams stopped being adversarial, not that the method
436    /// improved — so it asserts the breach explicitly rather than just comparing the two.
437    #[test]
438    fn split_conformal_breaches_where_the_e_process_holds() {
439        let alpha = 0.15;
440        let n = 6000;
441        let mut rng = Lcg(20_260_814);
442
443        // Phase 1 doubles as split-conformal's calibration set.
444        let mut calib: Vec<(f64, bool)> = Vec::new();
445        for _ in 0..1000 {
446            let score = 0.70 + rng.next_f64() * 0.30;
447            calib.push((score, correct_for(score, rng.next_f64())));
448        }
449        let fixed = crate::conformal::calibrate(&calib, alpha, 0.05, 30).threshold;
450
451        let mut e = EProcessRiskControl::new(alpha, 0.05, DEFAULT_BET, &grid());
452        // Warm the e-process on the same phase-1 data, so neither method gets a data advantage.
453        for &(s, c) in &calib {
454            e.observe_served(s, c);
455        }
456
457        let (mut sc_served, mut sc_fails) = (0u64, 0u64);
458        let (mut ep_served, mut ep_fails) = (0u64, 0u64);
459        for t in 0..n {
460            let score = 0.70 + rng.next_f64() * 0.30;
461            // Phase 2: the relationship between score and correctness collapses.
462            let correct = if t < n / 2 {
463                correct_for(score, rng.next_f64())
464            } else {
465                rng.next_f64() < 0.60
466            };
467            if score >= fixed {
468                sc_served += 1;
469                sc_fails += u64::from(!correct);
470            }
471            if e.should_serve(score) {
472                ep_served += 1;
473                ep_fails += u64::from(!correct);
474            }
475            e.observe_served(score, correct);
476        }
477
478        let sc_rate = sc_fails as f64 / sc_served.max(1) as f64;
479        let ep_rate = ep_fails as f64 / ep_served.max(1) as f64;
480        assert!(
481            sc_rate > alpha,
482            "the stream is meant to break split conformal, but its rate was {sc_rate:.3} <= alpha={alpha} \
483             — the test has stopped being adversarial and proves nothing"
484        );
485        assert!(
486            ep_rate < sc_rate,
487            "the e-process ({ep_rate:.3}) must beat split conformal ({sc_rate:.3}) under shift"
488        );
489    }
490
491    /// **The cap must bound how long a stale certification survives.**
492    ///
493    /// Written because the first mutation run exposed that nothing covered the cap: deleting it
494    /// left all other tests green. Without a ceiling, a long clean stretch banks unbounded surplus
495    /// evidence, and after the regime turns bad the threshold keeps serving until that surplus is
496    /// burned off — the bound still holds in the limit while realized failure sits above `alpha`
497    /// for thousands of rounds. That is the long-run-average failure mode wearing a new hat.
498    ///
499    /// Capped vs uncapped on an identical stream, measuring rounds-to-de-certify.
500    #[test]
501    fn the_cap_bounds_how_long_a_stale_certification_survives() {
502        let alpha = 0.10;
503        let lag = |cap: f64| -> u64 {
504            let mut e = EProcessRiskControl::with_cap(alpha, 0.05, DEFAULT_BET, cap, &grid());
505            let mut rng = Lcg(99);
506            // Long clean stretch: bank evidence.
507            for _ in 0..5000 {
508                let score = 0.95 + rng.next_f64() * 0.05;
509                e.observe_served(score, correct_for(score, rng.next_f64()));
510            }
511            assert!(
512                e.certified_threshold().is_some(),
513                "the clean stretch must certify, or the lag measurement is meaningless"
514            );
515            // Regime turns hostile: every served item now fails. Count rounds to de-certify.
516            let mut rounds = 0u64;
517            while e.certified_threshold().is_some() && rounds < 100_000 {
518                e.observe_served(0.99, false);
519                rounds += 1;
520            }
521            rounds
522        };
523
524        let capped = lag(DEFAULT_CAP);
525        let uncapped = lag(f64::INFINITY);
526        assert!(
527            capped < uncapped,
528            "the cap must shorten de-certification lag: capped={capped} uncapped={uncapped}"
529        );
530        assert!(
531            capped <= 100,
532            "a capped controller must drop a contradicted certification within ~tens of rounds, took {capped}"
533        );
534    }
535
536    /// **A large bet must not permanently annihilate a threshold.**
537    ///
538    /// Flagged in review. The failure factor is `1 + bet*(alpha - 1)`, which hits exactly zero at
539    /// `bet = 1/(1-alpha)` and goes negative past it. Zero is absorbing under multiplication: one
540    /// failure would retire that threshold forever, and no later evidence could revive it. The
541    /// controller would look like it was still learning while being permanently dead.
542    ///
543    /// `bet` is reachable through the public `with_cap` and through config, so it is validated
544    /// input, not a theoretical corner.
545    #[test]
546    fn an_oversized_bet_cannot_permanently_kill_a_threshold() {
547        let alpha = 0.20;
548        // bet = 5.0 is far past the annihilating 1/(1-0.2) = 1.25.
549        let mut e = EProcessRiskControl::with_cap(alpha, 0.05, 5.0, DEFAULT_CAP, &grid());
550        e.observe_served(0.9, false);
551        assert!(
552            e.e_values().iter().all(|v| *v > 0.0),
553            "a failure must shrink e-values, never zero them: {:?}",
554            e.e_values()
555        );
556
557        // And the process must still be able to recover and certify afterwards.
558        let mut rng = Lcg(7);
559        for _ in 0..4000 {
560            let score = 0.95 + rng.next_f64() * 0.05;
561            e.observe_served(score, correct_for(score, rng.next_f64()));
562        }
563        assert!(
564            e.certified_threshold().is_some(),
565            "a controller that took one early failure must still be able to certify later"
566        );
567    }
568
569    /// **Precedence contract with ACI**, verified at the source rather than trusted in prose.
570    ///
571    /// `proxy.rs` prefers the e-process threshold over `AdaptiveConformal`'s whenever one is
572    /// certified, and falls through otherwise. That is only safe because an uncertified controller
573    /// returns `None` rather than a default — if it ever returned a number before earning it, the
574    /// router would silently serve on an unproven threshold and outrank a method that at least has
575    /// a long-run guarantee.
576    #[test]
577    fn an_uncertified_controller_yields_nothing_for_the_router_to_prefer() {
578        let mut e = EProcessRiskControl::new(0.2, 0.05, DEFAULT_BET, &grid());
579        // Some evidence, but nowhere near the crossing level.
580        for _ in 0..10 {
581            e.observe_served(0.95, true);
582        }
583        assert!(
584            e.certified_threshold().is_none(),
585            "ten rounds must not certify at delta=0.05 over a 21-point grid"
586        );
587        // ...and once it HAS earned it, the router gets a real threshold to prefer.
588        let mut rng = Lcg(11);
589        for _ in 0..4000 {
590            let score = 0.95 + rng.next_f64() * 0.05;
591            e.observe_served(score, correct_for(score, rng.next_f64()));
592        }
593        let c = e
594            .certified_threshold()
595            .expect("sustained clean evidence must certify");
596        assert!(
597            (0.0..=1.0).contains(&c.threshold),
598            "a certified threshold must be servable: {}",
599            c.threshold
600        );
601    }
602
603    /// `delta = 0` means "zero error budget", and must certify NOTHING rather than everything.
604    ///
605    /// Review raised this as a possible false-certification path. It is already handled —
606    /// `crossing_level` returns infinity for a non-positive delta — but "already correct" is not a
607    /// reason to leave a fail-closed contract untested, and an infinity comparison is exactly the
608    /// kind of thing a later refactor breaks silently.
609    #[test]
610    fn a_zero_error_budget_certifies_nothing() {
611        for delta in [0.0, -0.1] {
612            let mut e = EProcessRiskControl::new(0.2, delta, DEFAULT_BET, &grid());
613            let mut rng = Lcg(3);
614            for _ in 0..5000 {
615                let score = 0.99;
616                e.observe_served(score, correct_for(score, rng.next_f64()));
617            }
618            assert!(
619                e.crossing_level().is_infinite(),
620                "delta={delta} must make the crossing level unreachable"
621            );
622            assert!(
623                e.certified_threshold().is_none(),
624                "delta={delta} is a zero error budget: nothing may ever be certified"
625            );
626            assert!(!e.should_serve(1.0), "and nothing may be served");
627        }
628    }
629
630    /// Bonferroni: the crossing level must scale with grid size. Testing every threshold at the
631    /// full `delta` is the multiple-testing trap — each e-process looks valid alone, while the
632    /// family-wise false-certification rate quietly grows with the grid.
633    #[test]
634    fn the_crossing_level_scales_with_grid_size() {
635        let small = EProcessRiskControl::new(0.1, 0.05, DEFAULT_BET, &[0.5, 0.6]);
636        let large = EProcessRiskControl::new(0.1, 0.05, DEFAULT_BET, &grid());
637        assert!(
638            large.crossing_level() > small.crossing_level(),
639            "a larger grid must demand more evidence per threshold: {} vs {}",
640            large.crossing_level(),
641            small.crossing_level()
642        );
643        assert!(
644            (small.crossing_level() - 2.0 / 0.05).abs() < 1e-9,
645            "crossing level must be exactly n/delta"
646        );
647    }
648
649    /// A threshold strict enough to have escalated an item learns nothing from it. Without this,
650    /// unused thresholds would accumulate evidence they never earned and certify on traffic they
651    /// would never have served.
652    #[test]
653    fn thresholds_that_would_not_have_served_are_not_updated() {
654        let mut e = EProcessRiskControl::new(0.1, 0.05, DEFAULT_BET, &[0.2, 0.9]);
655        e.observe_served(0.5, true);
656        let vals = e.e_values();
657        assert!(
658            (vals[1] - 1.0).abs() < f64::EPSILON,
659            "threshold 0.9 did not serve a 0.5-scoring item, so its e-value must stay 1.0, got {}",
660            vals[1]
661        );
662        assert!(
663            vals[0] > 1.0,
664            "threshold 0.2 did serve it correctly, so its e-value must grow, got {}",
665            vals[0]
666        );
667    }
668
669    /// A degenerate grid must be inert, not a panic. `new` takes a slice from config, and an empty
670    /// or non-finite grid is a plausible misconfiguration that must fail closed.
671    #[test]
672    fn a_degenerate_grid_certifies_nothing_and_does_not_panic() {
673        let mut e = EProcessRiskControl::new(0.1, 0.05, DEFAULT_BET, &[f64::NAN, f64::INFINITY]);
674        e.observe_served(0.9, true);
675        assert!(e.certified_threshold().is_none());
676        assert!(!e.should_serve(1.0));
677        assert!(
678            e.crossing_level().is_infinite(),
679            "an empty grid can never cross"
680        );
681    }
682}