pub struct AdaptiveConformal { /* private fields */ }Expand description
Online / adaptive conformal — Gibbs & Candès (2021), Adaptive Conformal Inference Under
Distribution Shift. calibrate fixes a threshold ONCE and assumes exchangeability; under real
drift (models change, prompts change, the gate’s error rate moves) the realized served-failure
wanders off target. AdaptiveConformal instead tracks the serving threshold online from
realized outcomes, so the long-run served-failure rate stays at alpha as the workload shifts.
This is the “gate that recalibrates itself from live feedback”: every deferred verdict nudges the
threshold, so it never drifts too loose (serving junk) or too strict (escalating needlessly). Feed
it the deferred-feedback stream and read AdaptiveConformal::threshold on the router hot path.
Implementations§
Source§impl AdaptiveConformal
impl AdaptiveConformal
Sourcepub fn new(alpha: f64, gamma: f64, init_threshold: f64) -> Self
pub fn new(alpha: f64, gamma: f64, init_threshold: f64) -> Self
alpha = target served-failure rate; gamma = step size (e.g. 0.01–0.05, larger tracks
shift faster but noisier); init_threshold = starting λ (e.g. a calibrate result).
Sourcepub fn should_serve(&self, score: f64) -> bool
pub fn should_serve(&self, score: f64) -> bool
Serve iff score ≥ the current threshold.
Sourcepub fn observe_served(&mut self, was_correct: bool)
pub fn observe_served(&mut self, was_correct: bool)
Observe a served item’s realized correctness (from deferred feedback) and adapt λ. The
ACI update raises λ when served errors exceed alpha (serve more conservatively) and lowers
it when they’re below (serve more), so realized served-failure converges to alpha.
Sourcepub fn realized_served_failure(&self) -> f64
pub fn realized_served_failure(&self) -> f64
Realized served-failure rate so far (running diagnostic).
Trait Implementations§
Source§impl Clone for AdaptiveConformal
impl Clone for AdaptiveConformal
Source§fn clone(&self) -> AdaptiveConformal
fn clone(&self) -> AdaptiveConformal
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source. Read more