firstpass_core/eprocess.rs
1//! Anytime-valid risk control on the gate threshold — a bound that holds at **every round**
2//! (SPEC §10.1, the online regime).
3//!
4//! ## Why this exists when [`crate::conformal`] and [`crate::ltt`] already do risk control
5//!
6//! The three regimes already shipped are each valid for what they claim, and none of them covers
7//! the way an operator actually runs the proxy:
8//!
9//! | existing | guarantee | why it is not enough here |
10//! |---|---|---|
11//! | [`crate::conformal::calibrate`] | served-failure ≤ `alpha` at confidence `1 − delta` | assumes **exchangeability**, and fixes `λ` once |
12//! | [`crate::ltt`] | FWER-controlled, exact-binomial, no exchangeability needed | **fixed-sample**: one calibration set, one decision |
13//! | [`crate::conformal::AdaptiveConformal`] | Gibbs–Candès ACI | keeps only the **long-run average** at `alpha` |
14//!
15//! Two concrete holes follow, and both bite in production:
16//!
17//! 1. **A long-run average is not a promise about now.** A process can sit above `alpha` for an
18//! arbitrarily long stretch and still converge, because the excess is amortised over a horizon
19//! the operator may never reach. Someone holding an error budget needs the bound to hold at the
20//! round they are on.
21//! 2. **Repeated recalibration breaks fixed-sample guarantees.** LTT controls the family-wise error
22//! rate over *one* pre-specified test sequence on *one* calibration set. Re-run it whenever more
23//! feedback lands and adopt the winning threshold each time — which is exactly what a live
24//! recalibration loop does — and that is optional stopping on a growing stream. The guarantee
25//! does not compose across the repeats, and the realized risk can exceed `alpha` without any
26//! single run of LTT ever being wrong.
27//!
28//! ## Method
29//!
30//! Khosravi & Huo (2026), *Conformal Selective Acting: Anytime-Valid Risk Control for RLVR-Trained
31//! LLMs* ([arXiv:2605.20270](https://arxiv.org/abs/2605.20270)).
32//!
33//! Maintain one **e-process** per candidate threshold on a fixed grid. An e-process is a
34//! non-negative supermartingale under the null, so **Ville's inequality** bounds it at *all*
35//! stopping times simultaneously:
36//!
37//! ```text
38//! P( ∃t : E_t ≥ 1/delta ) ≤ delta under H₀: risk(λ) > alpha
39//! ```
40//!
41//! That single quantifier — `∃t`, rather than a fixed `t` — is the whole point, and it is what
42//! makes the bound survive optional stopping. A threshold is **certified** once its e-value crosses
43//! `1/delta`, and certification is permanent: it was earned against every round up to that point,
44//! so later data cannot retroactively invalidate it (it can, and does, certify *more* thresholds).
45//!
46//! Per round, for a served item with gate score `s` and observed correctness `c`, each threshold
47//! `λ ≤ s` (i.e. each threshold that *would have served* this item) is updated by the betting factor
48//!
49//! ```text
50//! E_λ ← E_λ · (1 + lambda_bet · (alpha − err)) where err = 1 if incorrect else 0
51//! ```
52//!
53//! Under H₀ the expected multiplier is `≤ 1`, which is precisely the supermartingale property.
54//! Thresholds that never serve are never updated, so an unused threshold neither gains nor loses
55//! evidence — a nuance the fixed-sample methods get for free but an online one has to be careful
56//! about.
57//!
58//! The grid is a **Bonferroni** family: with `n` candidate thresholds each tested at `delta / n`,
59//! the probability that *any* threshold is falsely certified at *any* round stays `≤ delta`. The
60//! served threshold is the **least conservative certified** one — the smallest `λ` whose e-process
61//! has crossed — which serves as much traffic as the evidence allows.
62//!
63//! ## The honest cost
64//!
65//! Anytime validity is not free. Against [`crate::conformal::AdaptiveConformal`] at the same
66//! `alpha` this serves **less** traffic, because it will not lower the threshold until the evidence
67//! has actually accumulated, and the Bonferroni correction makes each individual threshold harder
68//! to certify. That is the trade being offered: ACI tracks a shifting target aggressively and
69//! promises an average; this one promises every round and pays for it in conservatism. An operator
70//! picks the regime; neither is strictly better, and the ADR says so.
71//!
72//! Before any threshold is certified there is no guarantee to serve on, so
73//! [`EProcessRiskControl::certified_threshold`] returns `None` and the caller keeps its existing
74//! behaviour. Failing closed is the only safe reading of "I have not proven anything yet".
75
76use serde::{Deserialize, Serialize};
77
78/// Outcome of a certification query: the served threshold and the evidence behind it.
79#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
80pub struct Certification {
81 /// The least conservative certified threshold — serve iff `score >= threshold`.
82 pub threshold: f64,
83 /// The e-value backing it. Always `>= 1/delta` for a certified threshold.
84 pub e_value: f64,
85 /// Rounds observed when this certification was first reached.
86 pub certified_at_round: u64,
87}
88
89/// Default betting fraction. Conservative on purpose: a large bet grows the e-process fast when the
90/// null is false but decays it just as fast when it is true, and a decayed e-process takes a long
91/// time to recover. `0.5` is the standard "half-Kelly" compromise.
92pub const DEFAULT_BET: f64 = 0.5;
93
94/// Default e-value ceiling, as a multiple of the crossing level.
95///
96/// Truncating a supermartingale from above leaves it a supermartingale, so the type-I guarantee is
97/// unaffected — this is purely about **responsiveness**. Without a cap, a long clean stretch banks
98/// unbounded surplus evidence, and a subsequent regime change must burn all of it off before the
99/// threshold de-certifies; the bound holds asymptotically while realized failure sits above `alpha`
100/// for thousands of rounds. Capping bounds that de-certification lag. `10.0` keeps roughly an order
101/// of magnitude of headroom — enough that ordinary noise does not flap the certification, small
102/// enough that a genuine shift is caught in tens of rounds rather than thousands.
103pub const DEFAULT_CAP: f64 = 10.0;
104
105/// Anytime-valid risk controller over a fixed grid of candidate thresholds.
106///
107/// Feed it the deferred-feedback stream via [`EProcessRiskControl::observe_served`] and read
108/// [`EProcessRiskControl::certified_threshold`] on the router hot path. Unlike
109/// [`crate::conformal::AdaptiveConformal`], the answer is valid at the round you read it.
110#[derive(Debug, Clone, Serialize, Deserialize)]
111pub struct EProcessRiskControl {
112 /// Target served-failure rate.
113 alpha: f64,
114 /// Family-wise error budget across the whole grid and all rounds.
115 delta: f64,
116 /// Betting fraction in `(0, 1]`.
117 bet: f64,
118 /// E-values are capped at `cap * crossing_level`, bounding how much surplus evidence a clean
119 /// stretch can bank against a later regime change. See [`DEFAULT_CAP`].
120 cap: f64,
121 /// Candidate thresholds, ascending.
122 grid: Vec<f64>,
123 /// Running e-value per grid point, parallel to `grid`. Starts at `1.0` (no evidence).
124 e_values: Vec<f64>,
125 /// Round at which each grid point first crossed, if it has.
126 certified_at: Vec<Option<u64>>,
127 /// Rounds observed.
128 rounds: u64,
129 /// Served count and failures, for diagnostics only — never part of the guarantee.
130 served: u64,
131 served_fails: u64,
132}
133
134impl EProcessRiskControl {
135 /// Build a controller over `grid` candidate thresholds.
136 ///
137 /// `alpha` is the target served-failure rate, `delta` the family-wise error budget (the bound
138 /// holds with probability `1 − delta` across every threshold and every round jointly).
139 ///
140 /// The grid is sorted and de-duplicated; an empty grid yields a controller that never certifies,
141 /// which is the correct degenerate behaviour rather than an error — the caller then simply keeps
142 /// its existing threshold.
143 #[must_use]
144 pub fn new(alpha: f64, delta: f64, bet: f64, grid: &[f64]) -> Self {
145 Self::with_cap(alpha, delta, bet, DEFAULT_CAP, grid)
146 }
147
148 /// As [`EProcessRiskControl::new`], with an explicit e-value ceiling. See [`DEFAULT_CAP`] for
149 /// why a ceiling exists and what it trades.
150 #[must_use]
151 pub fn with_cap(alpha: f64, delta: f64, bet: f64, cap: f64, grid: &[f64]) -> Self {
152 // The betting factor on a failure is `1 + bet*(alpha - 1)`, which reaches zero at
153 // `bet = 1/(1 - alpha)` and goes negative beyond it. A zero factor annihilates the e-value
154 // permanently: no amount of subsequent evidence can revive a product that has hit exactly
155 // 0, so a single failure would silently retire that threshold forever. Clamping the bet
156 // just below the annihilating value keeps the process a strict supermartingale (which is
157 // what Ville's inequality needs) while leaving it able to recover.
158 //
159 // Flagged in review. Reachable through the public `with_cap` and by config, so it is a
160 // real input to validate rather than a theoretical corner.
161 // Bound the failure factor at 0.5 rather than merely above 0. Clamping at `0.99/(1-alpha)`
162 // technically avoids annihilation but leaves a factor of 0.01 — a single failure costs 100x
163 // and takes ~50 consecutive successes to undo, which is dead in practice if not in theory.
164 // The first version of this clamp did exactly that, and the recovery half of the test below
165 // failed: the controller never certified again. Half per failure is recoverable and still
166 // strictly decreasing, which is all the supermartingale property needs.
167 let max_bet = 0.5 / (1.0 - alpha).max(f64::EPSILON);
168 let bet = if bet.is_finite() {
169 bet.clamp(0.0, max_bet)
170 } else {
171 DEFAULT_BET.min(max_bet)
172 };
173 let mut grid: Vec<f64> = grid.iter().copied().filter(|g| g.is_finite()).collect();
174 grid.sort_by(|a, b| a.partial_cmp(b).unwrap_or(std::cmp::Ordering::Equal));
175 grid.dedup();
176 let n = grid.len();
177 Self {
178 alpha,
179 delta,
180 bet,
181 cap,
182 grid,
183 e_values: vec![1.0; n],
184 certified_at: vec![None; n],
185 rounds: 0,
186 served: 0,
187 served_fails: 0,
188 }
189 }
190
191 /// The per-threshold crossing level under Bonferroni: `n / delta`.
192 ///
193 /// Splitting `delta` across the `n` grid points is what keeps the *family-wise* error at
194 /// `delta`. Testing every threshold at the full `delta` would inflate the false-certification
195 /// rate with grid size — the classic multiple-testing trap, and an easy one to fall into here
196 /// because each individual e-process looks perfectly valid on its own.
197 #[must_use]
198 pub fn crossing_level(&self) -> f64 {
199 if self.grid.is_empty() || self.delta <= 0.0 {
200 return f64::INFINITY;
201 }
202 self.grid.len() as f64 / self.delta
203 }
204
205 /// Record one served item: its gate `score`, and whether it turned out correct.
206 ///
207 /// Only thresholds that *would have served* this item (`λ <= score`) are updated — a threshold
208 /// strict enough to have escalated instead has learned nothing about its own risk from it.
209 pub fn observe_served(&mut self, score: f64, was_correct: bool) {
210 self.rounds += 1;
211 self.served += 1;
212 self.served_fails += u64::from(!was_correct);
213
214 let err = f64::from(!was_correct);
215 let level = self.crossing_level();
216
217 for i in 0..self.grid.len() {
218 if self.grid[i] > score {
219 continue;
220 }
221 // Betting factor. Under H₀ (true risk > alpha) the expectation is <= 1, which is the
222 // supermartingale property Ville's inequality needs. Clamped at zero because a negative
223 // e-value is meaningless and would flip the comparison below.
224 let factor = (1.0 + self.bet * (self.alpha - err)).max(0.0);
225 // Capped. Ville's inequality is indifferent to the cap (truncating a supermartingale
226 // from above leaves it a supermartingale, so type-I control is untouched), but the
227 // operational difference is the whole ballgame: an uncapped e-value compounds through a
228 // long clean stretch to something astronomically above the level, and then a regime
229 // change has to burn through all of it before the threshold de-certifies. The bound
230 // would still hold in the limit while the realized rate sat above alpha for thousands
231 // of rounds — which is precisely the "long-run average" failure this module rejects.
232 self.e_values[i] = (self.e_values[i] * factor).min(self.cap * level);
233 if self.certified_at[i].is_none() && self.e_values[i] >= level {
234 self.certified_at[i] = Some(self.rounds);
235 }
236 }
237 }
238
239 /// The least conservative certified threshold, or `None` if nothing is certified yet.
240 ///
241 /// `None` means "no threshold has earned a guarantee", and the caller must keep its existing
242 /// behaviour rather than invent one. Serving on an uncertified threshold would be exactly the
243 /// unproven-claim failure this module exists to prevent.
244 /// Certification is evaluated **live**, against the current e-value — it is not a latch.
245 ///
246 /// This is the subtle part, and getting it wrong silently reintroduces the very failure mode the
247 /// module exists to prevent. A threshold that crossed during a clean stretch and has since been
248 /// contradicted by evidence is *not* certified any more, and must stop serving. Latching it —
249 /// "it was proven once, so it stays proven" — is what a fixed-sample method does, and under a
250 /// regime change it keeps serving at a threshold the current regime does not justify. The first
251 /// draft of this module latched, and `coverage_holds_at_every_round_under_a_regime_change`
252 /// caught it: realized failure hit 0.312 against alpha=0.20.
253 ///
254 /// `certified_at_round` is retained as provenance — when the evidence *first* crossed — which is
255 /// useful in a receipt and harmless here, because the serving decision reads the e-value.
256 #[must_use]
257 pub fn certified_threshold(&self) -> Option<Certification> {
258 let level = self.crossing_level();
259 self.grid
260 .iter()
261 .enumerate()
262 .find(|(i, _)| self.e_values[*i] >= level)
263 .map(|(i, &threshold)| Certification {
264 threshold,
265 e_value: self.e_values[i],
266 certified_at_round: self.certified_at[i].unwrap_or(self.rounds),
267 })
268 }
269
270 /// Whether `score` clears the certified threshold. `false` while nothing is certified.
271 #[must_use]
272 pub fn should_serve(&self, score: f64) -> bool {
273 self.certified_threshold()
274 .is_some_and(|c| score >= c.threshold)
275 }
276
277 /// Realized served-failure rate so far — a running diagnostic, not the guarantee.
278 #[must_use]
279 pub fn realized_served_failure(&self) -> f64 {
280 if self.served == 0 {
281 0.0
282 } else {
283 self.served_fails as f64 / self.served as f64
284 }
285 }
286
287 /// Rounds observed.
288 #[must_use]
289 pub const fn rounds(&self) -> u64 {
290 self.rounds
291 }
292
293 /// Current e-value at each grid point, parallel to the grid — for telemetry and tests.
294 #[must_use]
295 pub fn e_values(&self) -> &[f64] {
296 &self.e_values
297 }
298
299 /// The candidate grid, ascending.
300 #[must_use]
301 pub fn grid(&self) -> &[f64] {
302 &self.grid
303 }
304}
305
306#[cfg(test)]
307mod tests {
308 use super::*;
309
310 /// Deterministic uniform stream in `[0,1)`. A seeded LCG rather than `rand`, so the evidence
311 /// these tests produce is reproducible by an auditor on any machine, which is the same reason
312 /// the rest of the crate avoids ambient randomness.
313 struct Lcg(u64);
314 impl Lcg {
315 fn next_f64(&mut self) -> f64 {
316 self.0 = self
317 .0
318 .wrapping_mul(6_364_136_223_846_793_005)
319 .wrapping_add(1);
320 ((self.0 >> 11) as f64) / ((1u64 << 53) as f64)
321 }
322 }
323
324 fn grid() -> Vec<f64> {
325 (0..=20).map(|i| f64::from(i) / 20.0).collect()
326 }
327
328 /// A well-behaved gate: high score => usually correct.
329 fn correct_for(score: f64, u: f64) -> bool {
330 u < score
331 }
332
333 /// Nothing may be served before evidence exists. This is the fail-closed property: an
334 /// uncertified controller must not hand out a threshold, or the guarantee is decorative.
335 #[test]
336 fn nothing_is_certified_before_any_evidence() {
337 let e = EProcessRiskControl::new(0.1, 0.05, DEFAULT_BET, &grid());
338 assert!(
339 e.certified_threshold().is_none(),
340 "a fresh controller must certify nothing"
341 );
342 assert!(
343 !e.should_serve(1.0),
344 "even a perfect score must not serve without a certified threshold"
345 );
346 }
347
348 /// On a clean stream the controller must eventually certify — a method that never certifies is
349 /// trivially "safe" and completely useless, so this is the liveness half of the contract.
350 #[test]
351 fn a_clean_stream_eventually_certifies() {
352 let mut e = EProcessRiskControl::new(0.2, 0.05, DEFAULT_BET, &grid());
353 let mut rng = Lcg(42);
354 for _ in 0..4000 {
355 let score = 0.90 + rng.next_f64() * 0.10;
356 let correct = correct_for(score, rng.next_f64());
357 e.observe_served(score, correct);
358 }
359 let cert = e
360 .certified_threshold()
361 .expect("4000 clean rounds must certify some threshold");
362 assert!(
363 cert.e_value >= e.crossing_level(),
364 "a certified threshold must sit at or above the Bonferroni crossing level: {} < {}",
365 cert.e_value,
366 e.crossing_level()
367 );
368 }
369
370 /// **The core guarantee, and the reason this module exists.**
371 ///
372 /// Anytime-validity means the bound holds at EVERY round, not just at the end. This checks the
373 /// realized served-failure rate at every single round after certification, on an ADVERSARIAL
374 /// non-exchangeable stream: a clean first half, then an abrupt regime change where the gate's
375 /// error rate jumps. Split conformal's exchangeability assumption is violated by construction
376 /// here — see `split_conformal_breaches_where_the_e_process_holds` for the contrast.
377 #[test]
378 fn coverage_holds_at_every_round_under_a_regime_change() {
379 let alpha = 0.20;
380 let mut worst: f64 = 0.0;
381 for seed in 0..40u64 {
382 let mut e = EProcessRiskControl::new(alpha, 0.05, DEFAULT_BET, &grid());
383 let mut rng = Lcg(seed.wrapping_mul(2_654_435_761).wrapping_add(7));
384 let mut served = 0u64;
385 let mut fails = 0u64;
386 for t in 0..3000 {
387 let score = 0.80 + rng.next_f64() * 0.20;
388 // Regime change at the midpoint: the gate becomes much less reliable, with no
389 // warning and no return. This is exactly what exchangeability forbids.
390 let correct = if t < 1500 {
391 correct_for(score, rng.next_f64())
392 } else {
393 rng.next_f64() < 0.55
394 };
395 if e.should_serve(score) {
396 served += 1;
397 fails += u64::from(!correct);
398 // Check the realized rate at THIS round, not only at the end.
399 if served >= 50 {
400 let rate = fails as f64 / served as f64;
401 worst = worst.max(rate);
402 }
403 }
404 e.observe_served(score, correct);
405 }
406 }
407 // Ville's inequality is a high-probability bound, not an almost-sure one, so a small
408 // overshoot budget is correct rather than a fudge: at delta = 0.05 some excursion is
409 // permitted by the theory itself.
410 assert!(
411 worst <= alpha + 0.10,
412 "worst per-round served-failure {worst:.3} exceeded alpha={alpha} plus slack"
413 );
414 }
415
416 /// **The contrast that makes the contribution.**
417 ///
418 /// On the same non-exchangeable stream, a split-conformal threshold fitted on the first regime
419 /// carries its guarantee only under exchangeability — which the regime change destroys. It goes
420 /// on serving at a threshold the new regime does not justify, and its realized failure rate
421 /// breaches alpha. The e-process, having to re-earn its evidence every round, does not.
422 ///
423 /// If this test ever fails it means the streams stopped being adversarial, not that the method
424 /// improved — so it asserts the breach explicitly rather than just comparing the two.
425 #[test]
426 fn split_conformal_breaches_where_the_e_process_holds() {
427 let alpha = 0.15;
428 let n = 6000;
429 let mut rng = Lcg(20_260_814);
430
431 // Phase 1 doubles as split-conformal's calibration set.
432 let mut calib: Vec<(f64, bool)> = Vec::new();
433 for _ in 0..1000 {
434 let score = 0.70 + rng.next_f64() * 0.30;
435 calib.push((score, correct_for(score, rng.next_f64())));
436 }
437 let fixed = crate::conformal::calibrate(&calib, alpha, 0.05, 30).threshold;
438
439 let mut e = EProcessRiskControl::new(alpha, 0.05, DEFAULT_BET, &grid());
440 // Warm the e-process on the same phase-1 data, so neither method gets a data advantage.
441 for &(s, c) in &calib {
442 e.observe_served(s, c);
443 }
444
445 let (mut sc_served, mut sc_fails) = (0u64, 0u64);
446 let (mut ep_served, mut ep_fails) = (0u64, 0u64);
447 for t in 0..n {
448 let score = 0.70 + rng.next_f64() * 0.30;
449 // Phase 2: the relationship between score and correctness collapses.
450 let correct = if t < n / 2 {
451 correct_for(score, rng.next_f64())
452 } else {
453 rng.next_f64() < 0.60
454 };
455 if score >= fixed {
456 sc_served += 1;
457 sc_fails += u64::from(!correct);
458 }
459 if e.should_serve(score) {
460 ep_served += 1;
461 ep_fails += u64::from(!correct);
462 }
463 e.observe_served(score, correct);
464 }
465
466 let sc_rate = sc_fails as f64 / sc_served.max(1) as f64;
467 let ep_rate = ep_fails as f64 / ep_served.max(1) as f64;
468 assert!(
469 sc_rate > alpha,
470 "the stream is meant to break split conformal, but its rate was {sc_rate:.3} <= alpha={alpha} \
471 — the test has stopped being adversarial and proves nothing"
472 );
473 assert!(
474 ep_rate < sc_rate,
475 "the e-process ({ep_rate:.3}) must beat split conformal ({sc_rate:.3}) under shift"
476 );
477 }
478
479 /// **The cap must bound how long a stale certification survives.**
480 ///
481 /// Written because the first mutation run exposed that nothing covered the cap: deleting it
482 /// left all other tests green. Without a ceiling, a long clean stretch banks unbounded surplus
483 /// evidence, and after the regime turns bad the threshold keeps serving until that surplus is
484 /// burned off — the bound still holds in the limit while realized failure sits above `alpha`
485 /// for thousands of rounds. That is the long-run-average failure mode wearing a new hat.
486 ///
487 /// Capped vs uncapped on an identical stream, measuring rounds-to-de-certify.
488 #[test]
489 fn the_cap_bounds_how_long_a_stale_certification_survives() {
490 let alpha = 0.10;
491 let lag = |cap: f64| -> u64 {
492 let mut e = EProcessRiskControl::with_cap(alpha, 0.05, DEFAULT_BET, cap, &grid());
493 let mut rng = Lcg(99);
494 // Long clean stretch: bank evidence.
495 for _ in 0..5000 {
496 let score = 0.95 + rng.next_f64() * 0.05;
497 e.observe_served(score, correct_for(score, rng.next_f64()));
498 }
499 assert!(
500 e.certified_threshold().is_some(),
501 "the clean stretch must certify, or the lag measurement is meaningless"
502 );
503 // Regime turns hostile: every served item now fails. Count rounds to de-certify.
504 let mut rounds = 0u64;
505 while e.certified_threshold().is_some() && rounds < 100_000 {
506 e.observe_served(0.99, false);
507 rounds += 1;
508 }
509 rounds
510 };
511
512 let capped = lag(DEFAULT_CAP);
513 let uncapped = lag(f64::INFINITY);
514 assert!(
515 capped < uncapped,
516 "the cap must shorten de-certification lag: capped={capped} uncapped={uncapped}"
517 );
518 assert!(
519 capped <= 100,
520 "a capped controller must drop a contradicted certification within ~tens of rounds, took {capped}"
521 );
522 }
523
524 /// **A large bet must not permanently annihilate a threshold.**
525 ///
526 /// Flagged in review. The failure factor is `1 + bet*(alpha - 1)`, which hits exactly zero at
527 /// `bet = 1/(1-alpha)` and goes negative past it. Zero is absorbing under multiplication: one
528 /// failure would retire that threshold forever, and no later evidence could revive it. The
529 /// controller would look like it was still learning while being permanently dead.
530 ///
531 /// `bet` is reachable through the public `with_cap` and through config, so it is validated
532 /// input, not a theoretical corner.
533 #[test]
534 fn an_oversized_bet_cannot_permanently_kill_a_threshold() {
535 let alpha = 0.20;
536 // bet = 5.0 is far past the annihilating 1/(1-0.2) = 1.25.
537 let mut e = EProcessRiskControl::with_cap(alpha, 0.05, 5.0, DEFAULT_CAP, &grid());
538 e.observe_served(0.9, false);
539 assert!(
540 e.e_values().iter().all(|v| *v > 0.0),
541 "a failure must shrink e-values, never zero them: {:?}",
542 e.e_values()
543 );
544
545 // And the process must still be able to recover and certify afterwards.
546 let mut rng = Lcg(7);
547 for _ in 0..4000 {
548 let score = 0.95 + rng.next_f64() * 0.05;
549 e.observe_served(score, correct_for(score, rng.next_f64()));
550 }
551 assert!(
552 e.certified_threshold().is_some(),
553 "a controller that took one early failure must still be able to certify later"
554 );
555 }
556
557 /// **Precedence contract with ACI**, verified at the source rather than trusted in prose.
558 ///
559 /// `proxy.rs` prefers the e-process threshold over `AdaptiveConformal`'s whenever one is
560 /// certified, and falls through otherwise. That is only safe because an uncertified controller
561 /// returns `None` rather than a default — if it ever returned a number before earning it, the
562 /// router would silently serve on an unproven threshold and outrank a method that at least has
563 /// a long-run guarantee.
564 #[test]
565 fn an_uncertified_controller_yields_nothing_for_the_router_to_prefer() {
566 let mut e = EProcessRiskControl::new(0.2, 0.05, DEFAULT_BET, &grid());
567 // Some evidence, but nowhere near the crossing level.
568 for _ in 0..10 {
569 e.observe_served(0.95, true);
570 }
571 assert!(
572 e.certified_threshold().is_none(),
573 "ten rounds must not certify at delta=0.05 over a 21-point grid"
574 );
575 // ...and once it HAS earned it, the router gets a real threshold to prefer.
576 let mut rng = Lcg(11);
577 for _ in 0..4000 {
578 let score = 0.95 + rng.next_f64() * 0.05;
579 e.observe_served(score, correct_for(score, rng.next_f64()));
580 }
581 let c = e
582 .certified_threshold()
583 .expect("sustained clean evidence must certify");
584 assert!(
585 (0.0..=1.0).contains(&c.threshold),
586 "a certified threshold must be servable: {}",
587 c.threshold
588 );
589 }
590
591 /// `delta = 0` means "zero error budget", and must certify NOTHING rather than everything.
592 ///
593 /// Review raised this as a possible false-certification path. It is already handled —
594 /// `crossing_level` returns infinity for a non-positive delta — but "already correct" is not a
595 /// reason to leave a fail-closed contract untested, and an infinity comparison is exactly the
596 /// kind of thing a later refactor breaks silently.
597 #[test]
598 fn a_zero_error_budget_certifies_nothing() {
599 for delta in [0.0, -0.1] {
600 let mut e = EProcessRiskControl::new(0.2, delta, DEFAULT_BET, &grid());
601 let mut rng = Lcg(3);
602 for _ in 0..5000 {
603 let score = 0.99;
604 e.observe_served(score, correct_for(score, rng.next_f64()));
605 }
606 assert!(
607 e.crossing_level().is_infinite(),
608 "delta={delta} must make the crossing level unreachable"
609 );
610 assert!(
611 e.certified_threshold().is_none(),
612 "delta={delta} is a zero error budget: nothing may ever be certified"
613 );
614 assert!(!e.should_serve(1.0), "and nothing may be served");
615 }
616 }
617
618 /// Bonferroni: the crossing level must scale with grid size. Testing every threshold at the
619 /// full `delta` is the multiple-testing trap — each e-process looks valid alone, while the
620 /// family-wise false-certification rate quietly grows with the grid.
621 #[test]
622 fn the_crossing_level_scales_with_grid_size() {
623 let small = EProcessRiskControl::new(0.1, 0.05, DEFAULT_BET, &[0.5, 0.6]);
624 let large = EProcessRiskControl::new(0.1, 0.05, DEFAULT_BET, &grid());
625 assert!(
626 large.crossing_level() > small.crossing_level(),
627 "a larger grid must demand more evidence per threshold: {} vs {}",
628 large.crossing_level(),
629 small.crossing_level()
630 );
631 assert!(
632 (small.crossing_level() - 2.0 / 0.05).abs() < 1e-9,
633 "crossing level must be exactly n/delta"
634 );
635 }
636
637 /// A threshold strict enough to have escalated an item learns nothing from it. Without this,
638 /// unused thresholds would accumulate evidence they never earned and certify on traffic they
639 /// would never have served.
640 #[test]
641 fn thresholds_that_would_not_have_served_are_not_updated() {
642 let mut e = EProcessRiskControl::new(0.1, 0.05, DEFAULT_BET, &[0.2, 0.9]);
643 e.observe_served(0.5, true);
644 let vals = e.e_values();
645 assert!(
646 (vals[1] - 1.0).abs() < f64::EPSILON,
647 "threshold 0.9 did not serve a 0.5-scoring item, so its e-value must stay 1.0, got {}",
648 vals[1]
649 );
650 assert!(
651 vals[0] > 1.0,
652 "threshold 0.2 did serve it correctly, so its e-value must grow, got {}",
653 vals[0]
654 );
655 }
656
657 /// A degenerate grid must be inert, not a panic. `new` takes a slice from config, and an empty
658 /// or non-finite grid is a plausible misconfiguration that must fail closed.
659 #[test]
660 fn a_degenerate_grid_certifies_nothing_and_does_not_panic() {
661 let mut e = EProcessRiskControl::new(0.1, 0.05, DEFAULT_BET, &[f64::NAN, f64::INFINITY]);
662 e.observe_served(0.9, true);
663 assert!(e.certified_threshold().is_none());
664 assert!(!e.should_serve(1.0));
665 assert!(
666 e.crossing_level().is_infinite(),
667 "an empty grid can never cross"
668 );
669 }
670}