Skip to main content

firstpass_proxy/
bandit.rs

1//! UCB1 start-rung bandit: predict-to-start, verify-to-serve.
2//!
3//! # Science
4//!
5//! Every request currently starts the escalation ladder at rung 0 (cheapest model). Many
6//! contexts (task kind × prompt-size bucket) have a learned empirical pass rate per rung: if
7//! rung 0 almost never passes for a given context, always starting there wastes money. This
8//! module learns a per-context start rung that minimises expected cost while the enforce gate
9//! still verifies the chosen rung's output before serving.
10//!
11//! **The invariant (predict-to-start, verify-to-serve):** the bandit only controls where the
12//! ladder *starts*. Gating, escalation, budget, and speculation are untouched. If the predicted
13//! start rung's output fails the gate, the ladder continues upward as normal — no downward
14//! retry. Prediction errors can cost money (latency, slightly higher expected cost when the
15//! estimate is wrong) but can never cause a wrong answer to be served.
16//!
17//! # Algorithm
18//!
19//! UCB1 (Auer et al. 2002) applied to cost-sensitive start-rung selection. For each candidate
20//! start s, the expected cost is:
21//!
22//! ```text
23//! E[s] = Σ_{r=s}^{top-1}  price_r · P(reach r | start s)
24//! P(reach r | start s) = Π_{q=s}^{r-1} (1 − p̂_q)
25//! p̂_q = clamp(successes_q / n_q  +  c · √(ln N / n_q),  0, 1)
26//! ```
27//!
28//! where `N` is the total gate-verdict observations in this context, `n_q` is the observations
29//! at rung `q`, and `c` is the exploration constant (default 1.0). Tie → lower s (conservative).
30//!
31//! **Prices are evaluated at the request's own size**, derived from the context's prompt-token
32//! band ([`ContextBucket::representative_prompt_tokens`]). This used to use fixed nominal tokens
33//! on the reasoning that "relative prices are what matters" — which measurement disproved.
34//!
35//! For two rungs the argmin above reduces to `start cheap ⟺ c₀ < p·c₁`, so what the cheap attempt
36//! costs *relative to what it can save* is the entire decision. With a constant token count that
37//! ratio is identical for every request, and the rule degenerates to a function of `p` alone.
38//! That throws away the signal that makes it pay: on 974-task MBPP, escalation is **adversely
39//! selected ~2.16×** — requests that fail the gate cost ~2.2× more at the cheap rung *and* ~2.1×
40//! more at the top rung, because hard requests are long requests. Offline replay of the measured
41//! matrix put the corrected rule 22% below plain first-pass at slightly *higher* accuracy. Sizing
42//! the price at the actual request is what lets this bandit see that.
43//!
44//! **Cold-start safety:** if the context has fewer than `min_observations` total gate verdicts,
45//! the bandit returns rung 0 (byte-identical to today's behavior). Abstain verdicts are not
46//! counted (only clear Pass/Fail gate outcomes are informative).
47//!
48//! # State
49//!
50//! ponytail: in-memory `Mutex`, per-process. Survives restarts via warm-start replay of the
51//! trace store at startup. No persistence of its own — the trace store is the durable record.
52
53use std::collections::HashMap;
54
55use firstpass_core::{Features, PriceTable, TaskKind, Verdict};
56
57/// The context bucket that keys the bandit's per-arm statistics.
58///
59/// Coarsened to keep the arm count dense: `prompt_bucket_coarse = prompt_token_bucket / 2`
60/// halves the bucket resolution (floor(log2(n)) → floor(log2(n)/2) in effect), trading
61/// granularity for faster warm-up.
62///
63/// ponytail: the ladder identity is not part of the key — in practice a given (TaskKind,
64/// prompt size) routes to the same ladder via the first-match routing rule. Would need to be
65/// included if operators run overlapping routes for the same context.
66#[derive(Debug, Clone, PartialEq, Eq, Hash)]
67pub struct ContextBucket {
68    /// Coarse task classification from the request feature vector.
69    pub task_kind: TaskKind,
70    /// `features.prompt_token_bucket / 2` — halved for denser arms.
71    pub prompt_bucket_coarse: u32,
72}
73
74impl ContextBucket {
75    /// Derive a context bucket from a request's feature vector.
76    #[must_use]
77    pub fn from_features(f: &Features) -> Self {
78        Self {
79            task_kind: f.task_kind,
80            prompt_bucket_coarse: f.prompt_token_bucket / 2,
81        }
82    }
83
84    /// A representative prompt size for this bucket, in tokens.
85    ///
86    /// `features::token_bucket` is `floor(log2(n))` and this key halves it, so the bucket covers
87    /// `[2^(2b), 2^(2b+2))`. The geometric midpoint of that band is `2^(2b+1)`, which is the
88    /// least-wrong single number to stand in for it — and the ordering that matters
89    /// (bigger bucket ⇒ bigger estimate) is exactly preserved.
90    ///
91    /// This exists because start-rung selection needs the request's *size*, not just its shape.
92    /// The raw count is deliberately never stored (privacy), but the band already is, and the band
93    /// carries the signal.
94    #[must_use]
95    pub fn representative_prompt_tokens(&self) -> u64 {
96        // Clamp the BAND before scaling it: clamping the exponent afterwards still overflows the
97        // multiply on a pathological bucket (caught by test, in debug where it panics rather than
98        // wrapping to a tiny shift in release).
99        1u64 << (self.prompt_bucket_coarse.min(9) * 2 + 1)
100    }
101}
102
103/// Per-(context, rung) gate-verdict counts. Abstain is excluded (see module doc).
104#[derive(Debug, Default, Clone)]
105struct ArmCounts {
106    pass: f64,
107    fail: f64,
108}
109
110impl ArmCounts {
111    fn n(&self) -> f64 {
112        self.pass + self.fail
113    }
114}
115
116/// Online UCB1 bandit for start-rung selection.
117///
118/// Keyed by [`ContextBucket`]; per context, tracks per-rung pass/fail counts of gate verdicts.
119/// Thread-safety comes from the caller wrapping this in `Arc<std::sync::Mutex<_>>` (mirroring
120/// `AdaptiveConformal` — both are per-process, in-memory, low-contention).
121#[derive(Debug)]
122pub struct StartRungBandit {
123    /// UCB1 exploration constant `c` ≥ 0 (used by [`Algorithm::Ucb1`]).
124    exploration: f64,
125    /// Cold-start threshold: contexts with fewer total observations return rung 0.
126    min_observations: usize,
127    /// Selection algorithm: deterministic UCB1 (auditable) or Thompson sampling (stochastic,
128    /// native propensities, better under non-stationarity with `discount < 1`).
129    algorithm: Algorithm,
130    /// Per-observation multiplicative decay of a context's counts, in `(0, 1]`. `1.0` = no
131    /// forgetting (stationary). Applied on every observe, so a context that stops matching
132    /// reality fades at rate `discount^n` (discounted Thompson sampling).
133    discount: f64,
134    /// xorshift64* PRNG state for Thompson draws (seeded once; deterministic in tests).
135    rng: u64,
136    /// context → (rung index → counts).
137    data: HashMap<ContextBucket, HashMap<u32, ArmCounts>>,
138    /// Pseudo-count strength for [`Self::choose_start_with_prior`]'s Beta blend: `strength ·
139    /// prior[r]` prior "pass" pseudo-observations and `strength · (1 − prior[r])` prior "fail"
140    /// pseudo-observations per rung, before any real observation. Irrelevant unless a prior is
141    /// actually passed in. Set via [`Self::with_prior_strength`]; mirrors
142    /// `PriorConfig::strength`'s default.
143    prior_strength: f64,
144}
145
146/// Start-rung selection algorithm.
147/// Monte-Carlo repetitions for the Thompson propensity estimate.
148const PROPENSITY_SAMPLES: usize = 64;
149
150/// The expected-cost argmin over candidate start rungs, shared by UCB1 and Thompson: walk the
151/// ladder from each candidate start, accumulating `P(reach r) · price(r)` with
152/// `P(reach r+1) = P(reach r) · (1 − p_pass(r))`. Ties prefer the lower start (conservative).
153fn argmin_expected_cost(
154    ladder: &[String],
155    prices: &PriceTable,
156    in_tokens: u64,
157    mut p_pass: impl FnMut(u32) -> f64,
158) -> u32 {
159    // Output is estimated at half the prompt, preserving the ratio the previous nominal constants
160    // used (1000 in / 500 out) while letting both scale with the actual request.
161    let out_tokens = (in_tokens / 2).max(1);
162
163    let mut best_s = 0u32;
164    let mut best_cost = f64::MAX;
165    for s in 0..ladder.len() {
166        let mut expected_cost = 0.0_f64;
167        let mut p_reach = 1.0_f64;
168        for (r, model) in ladder.iter().enumerate().skip(s) {
169            let price = prices
170                .get(model)
171                .map(|p| p.cost(in_tokens, out_tokens))
172                .unwrap_or(0.0);
173            expected_cost += p_reach * price;
174            p_reach *= 1.0 - p_pass(r as u32);
175            if p_reach < 1e-10 {
176                break;
177            }
178        }
179        if expected_cost < best_cost {
180            best_cost = expected_cost;
181            best_s = s as u32;
182        }
183    }
184    best_s
185}
186
187#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
188pub enum Algorithm {
189    /// Deterministic optimism (Auer et al. 2002): auditable, but propensities are degenerate
190    /// (0/1) — off-policy estimates then need the epsilon-greedy overlay.
191    #[default]
192    Ucb1,
193    /// Thompson sampling over Beta posteriors of gate-pass: stochastic by nature, so every
194    /// decision carries a non-degenerate propensity (Monte-Carlo estimated), and `discount`
195    /// handles model churn (Chapelle & Li 2011; discounted TS for non-stationarity).
196    Thompson,
197}
198
199impl StartRungBandit {
200    /// Create a new UCB1 bandit with the given cold-start and exploration settings.
201    #[must_use]
202    pub fn new(min_observations: usize, exploration: f64) -> Self {
203        Self::with_algorithm(min_observations, exploration, Algorithm::Ucb1, 1.0, 0x9E37)
204    }
205
206    /// Create a bandit with an explicit algorithm, discount, and PRNG seed (seed matters only
207    /// for [`Algorithm::Thompson`]; pass anything non-zero).
208    #[must_use]
209    pub fn with_algorithm(
210        min_observations: usize,
211        exploration: f64,
212        algorithm: Algorithm,
213        discount: f64,
214        seed: u64,
215    ) -> Self {
216        Self {
217            exploration,
218            min_observations,
219            algorithm,
220            discount: discount.clamp(f64::MIN_POSITIVE, 1.0),
221            rng: seed.max(1),
222            data: HashMap::new(),
223            prior_strength: 10.0,
224        }
225    }
226
227    /// Set the pseudo-count strength used by [`Self::choose_start_with_prior`] (builder-style).
228    #[must_use]
229    pub fn with_prior_strength(mut self, strength: f64) -> Self {
230        self.prior_strength = strength;
231        self
232    }
233
234    /// Apply one step of multiplicative forgetting to every arm in `ctx` (no-op at 1.0).
235    fn discount_context(&mut self, ctx: &ContextBucket) {
236        if self.discount >= 1.0 {
237            return;
238        }
239        if let Some(arms) = self.data.get_mut(ctx) {
240            for c in arms.values_mut() {
241                c.pass *= self.discount;
242                c.fail *= self.discount;
243            }
244        }
245    }
246
247    /// Next xorshift64* draw as a uniform in `[0, 1)`.
248    fn next_u01(&mut self) -> f64 {
249        let mut x = self.rng;
250        x ^= x >> 12;
251        x ^= x << 25;
252        x ^= x >> 27;
253        self.rng = x;
254        let v = x.wrapping_mul(0x2545_F491_4F6C_DD1D);
255        (v >> 11) as f64 / (1u64 << 53) as f64
256    }
257
258    /// Standard normal via Box–Muller on the internal PRNG.
259    fn next_normal(&mut self) -> f64 {
260        let u1 = self.next_u01().max(f64::MIN_POSITIVE);
261        let u2 = self.next_u01();
262        (-2.0 * u1.ln()).sqrt() * (std::f64::consts::TAU * u2).cos()
263    }
264
265    /// Gamma(shape ≥ 1, scale 1) sample — Marsaglia–Tsang squeeze method.
266    fn next_gamma(&mut self, shape: f64) -> f64 {
267        debug_assert!(shape >= 1.0, "Beta(+1 prior) keeps shapes >= 1");
268        let d = shape - 1.0 / 3.0;
269        let c = 1.0 / (9.0 * d).sqrt();
270        loop {
271            let x = self.next_normal();
272            let v = (1.0 + c * x).powi(3);
273            if v <= 0.0 {
274                continue;
275            }
276            let u = self.next_u01();
277            if u < 1.0 - 0.0331 * x.powi(4) || u.ln() < 0.5 * x * x + d * (1.0 - v + v.ln()) {
278                return d * v;
279            }
280        }
281    }
282
283    /// One Beta(pass+1, fail+1) posterior draw of the gate-pass probability for `(ctx, rung)`.
284    /// Unobserved arms draw from the uniform prior Beta(1, 1).
285    fn thompson_pass(&mut self, ctx: &ContextBucket, rung: u32) -> f64 {
286        let (a, b) = self
287            .data
288            .get(ctx)
289            .and_then(|arms| arms.get(&rung))
290            .map_or((1.0, 1.0), |c| (c.pass + 1.0, c.fail + 1.0));
291        let x = self.next_gamma(a);
292        let y = self.next_gamma(b);
293        x / (x + y)
294    }
295
296    /// Posterior-mean gate-pass estimate for `(ctx, rung)` — `None` when the context is cold
297    /// (below `min_observations`) or the arm is unobserved. Deterministic; used by the
298    /// speculative-deferral band, never for serving decisions.
299    #[must_use]
300    pub fn pass_estimate(&self, ctx: &ContextBucket, rung: u32) -> Option<f64> {
301        let arms = self.data.get(ctx)?;
302        let total: f64 = arms.values().map(ArmCounts::n).sum();
303        if total < self.min_observations as f64 {
304            return None;
305        }
306        let c = arms.get(&rung)?;
307        if c.n() == 0.0 {
308            return None;
309        }
310        Some((c.pass + 1.0) / (c.n() + 2.0))
311    }
312
313    /// Record a gate verdict for `(context, rung)`.
314    ///
315    /// `Abstain` is ignored — only clear Pass/Fail outcomes are informative for cost modelling.
316    pub fn observe(&mut self, ctx: &ContextBucket, rung: u32, verdict: Verdict) {
317        match verdict {
318            Verdict::Abstain => {} // not counted
319            Verdict::Pass => {
320                self.discount_context(ctx);
321                self.data
322                    .entry(ctx.clone())
323                    .or_default()
324                    .entry(rung)
325                    .or_default()
326                    .pass += 1.0;
327            }
328            Verdict::Fail => {
329                self.discount_context(ctx);
330                self.data
331                    .entry(ctx.clone())
332                    .or_default()
333                    .entry(rung)
334                    .or_default()
335                    .fail += 1.0;
336            }
337        }
338    }
339
340    /// UCB1-optimistic pass-rate estimate for `rung` in `ctx`.
341    ///
342    /// Returns 1.0 for unobserved rungs (optimistic exploration, UCB → ∞, clamped).
343    fn ucb_pass(&self, ctx: &ContextBucket, rung: u32, ln_n: f64) -> f64 {
344        let Some(arms) = self.data.get(ctx) else {
345            return 1.0; // context unseen: fully optimistic
346        };
347        let Some(counts) = arms.get(&rung) else {
348            return 1.0; // rung unobserved for this context: fully optimistic
349        };
350        let n = counts.n();
351        if n == 0.0 {
352            return 1.0;
353        }
354        let p_hat = counts.pass / n;
355        (p_hat + self.exploration * (ln_n / n).sqrt()).clamp(0.0, 1.0)
356    }
357
358    /// Choose the start rung that minimises expected cost for this context and ladder.
359    ///
360    /// Returns `0` in all cold-start cases:
361    /// - bandit is cold (total observations < `min_observations` for this context)
362    /// - `ladder` is empty
363    ///
364    /// Never returns a value ≥ `ladder.len()`.
365    #[must_use]
366    pub fn choose_start(&self, ctx: &ContextBucket, ladder: &[String], prices: &PriceTable) -> u32 {
367        let top = ladder.len();
368        if top == 0 {
369            return 0;
370        }
371
372        // Cold-start guard: total gate observations in this context.
373        let n_total: f64 = self
374            .data
375            .get(ctx)
376            .map(|arms| arms.values().map(ArmCounts::n).sum())
377            .unwrap_or(0.0);
378        if n_total < self.min_observations as f64 {
379            return 0;
380        }
381
382        let ln_n = n_total.ln();
383        argmin_expected_cost(ladder, prices, ctx.representative_prompt_tokens(), |r| {
384            self.ucb_pass(ctx, r, ln_n)
385        })
386    }
387
388    /// Choose the start rung and its selection propensity.
389    ///
390    /// - [`Algorithm::Ucb1`]: deterministic — returns `(choice, None)`; the epsilon-greedy
391    ///   overlay (if configured) supplies the logged propensity, exactly as before.
392    /// - [`Algorithm::Thompson`]: one posterior draw per rung drives the same expected-cost
393    ///   argmin as UCB1 (the decision), and the propensity is Monte-Carlo estimated by
394    ///   re-running the draw `PROPENSITY_SAMPLES` times — the standard estimator for TS
395    ///   selection probabilities (exact computation is analytically intractable). Cold-start
396    ///   contexts return `(0, None)` (deterministic, no propensity to log).
397    #[must_use]
398    pub fn choose_start_with_propensity(
399        &mut self,
400        ctx: &ContextBucket,
401        ladder: &[String],
402        prices: &PriceTable,
403    ) -> (u32, Option<f64>) {
404        if self.algorithm == Algorithm::Ucb1 {
405            return (self.choose_start(ctx, ladder, prices), None);
406        }
407        let top = ladder.len();
408        if top == 0 {
409            return (0, None);
410        }
411        let n_total: f64 = self
412            .data
413            .get(ctx)
414            .map(|arms| arms.values().map(ArmCounts::n).sum())
415            .unwrap_or(0.0);
416        if n_total < self.min_observations as f64 {
417            return (0, None);
418        }
419
420        let draw = |this: &mut Self| {
421            // One full posterior draw per rung, then the shared cost argmin.
422            let samples: Vec<f64> = (0..top)
423                .map(|r| this.thompson_pass(ctx, r as u32))
424                .collect();
425            argmin_expected_cost(ladder, prices, ctx.representative_prompt_tokens(), |r| {
426                samples[r as usize]
427            })
428        };
429
430        let choice = draw(self);
431        let matches = (0..PROPENSITY_SAMPLES)
432            .filter(|_| draw(self) == choice)
433            .count();
434        // Clamp away 0: the MC estimate can miss a genuinely-possible arm in finite samples,
435        // and a zero propensity would blow up IPS. 1/(2·M) is the standard floor.
436        let p = (matches as f64 / PROPENSITY_SAMPLES as f64)
437            .max(1.0 / (2.0 * PROPENSITY_SAMPLES as f64));
438        (choice, Some(p))
439    }
440
441    /// Raw observed (pass, fail) counts for `(ctx, rung)`, or `(0.0, 0.0)` if unobserved.
442    fn counts(&self, ctx: &ContextBucket, rung: u32) -> (f64, f64) {
443        self.data
444            .get(ctx)
445            .and_then(|arms| arms.get(&rung))
446            .map_or((0.0, 0.0), |c| (c.pass, c.fail))
447    }
448
449    /// Choose the start rung using an optional decision-model prior (verified predictive
450    /// routing), blended with any observed gate verdicts as Beta pseudo-counts.
451    ///
452    /// `prior[r]` is the cumulative gate-pass probability for rung `r`, in ladder order (see
453    /// [`firstpass_core::cumulative_pass`]) — one entry per ladder rung. Each rung gets
454    /// `prior_strength · prior[r]` prior "pass" and `prior_strength · (1 − prior[r])` prior
455    /// "fail" pseudo-observations, blended with whatever has actually been observed for this
456    /// context. Unlike [`Self::choose_start`]/[`Self::choose_start_with_propensity`], there is
457    /// **no cold-start cliff**: a context with zero real observations still gets an informed
458    /// answer straight from the prior.
459    ///
460    /// `prior = None`, an empty prior, or a length mismatch with `ladder` all fall back to
461    /// [`Self::choose_start_with_propensity`] unchanged — behavior is provably identical to
462    /// today when no prior is supplied.
463    #[must_use]
464    pub fn choose_start_with_prior(
465        &mut self,
466        ctx: &ContextBucket,
467        ladder: &[String],
468        prices: &PriceTable,
469        prior: Option<&[f64]>,
470    ) -> (u32, Option<f64>) {
471        let Some(prior) = prior else {
472            return self.choose_start_with_propensity(ctx, ladder, prices);
473        };
474        if prior.is_empty() || prior.len() != ladder.len() {
475            return self.choose_start_with_propensity(ctx, ladder, prices);
476        }
477        let strength = self.prior_strength;
478
479        if self.algorithm == Algorithm::Ucb1 {
480            let choice =
481                argmin_expected_cost(ladder, prices, ctx.representative_prompt_tokens(), |r| {
482                    let (obs_pass, obs_fail) = self.counts(ctx, r);
483                    let a = strength * prior[r as usize] + obs_pass;
484                    let b = strength * (1.0 - prior[r as usize]) + obs_fail;
485                    (a / (a + b)).clamp(0.0, 1.0)
486                });
487            return (choice, None);
488        }
489
490        let draw = |this: &mut Self| {
491            let samples: Vec<f64> = (0..ladder.len())
492                .map(|r| {
493                    let (obs_pass, obs_fail) = this.counts(ctx, r as u32);
494                    // +1.0 Laplace floor keeps both Beta shapes >= 1.0 (required by the Gamma
495                    // sampler) even when the prior is exactly 0 or 1 at this rung.
496                    let a = (strength * prior[r] + obs_pass + 1.0).max(1.0);
497                    let b = (strength * (1.0 - prior[r]) + obs_fail + 1.0).max(1.0);
498                    let x = this.next_gamma(a);
499                    let y = this.next_gamma(b);
500                    x / (x + y)
501                })
502                .collect();
503            argmin_expected_cost(ladder, prices, ctx.representative_prompt_tokens(), |r| {
504                samples[r as usize]
505            })
506        };
507
508        let choice = draw(self);
509        let matches = (0..PROPENSITY_SAMPLES)
510            .filter(|_| draw(self) == choice)
511            .count();
512        let p = (matches as f64 / PROPENSITY_SAMPLES as f64)
513            .max(1.0 / (2.0 * PROPENSITY_SAMPLES as f64));
514        (choice, Some(p))
515    }
516
517    /// Feed all attempts from a stored trace into this bandit — used for warm-start at startup.
518    pub fn feed_trace_attempts(
519        &mut self,
520        ctx: &ContextBucket,
521        attempts: &[firstpass_core::Attempt],
522    ) {
523        for attempt in attempts {
524            self.observe(ctx, attempt.rung, attempt.verdict);
525        }
526    }
527}
528
529#[cfg(test)]
530mod tests {
531    use super::*;
532    use firstpass_core::{
533        Attempt, Features, FinalOutcome, GENESIS_HASH, Mode, PolicyRef, RequestInfo, ServedFrom,
534        TaskKind, Trace,
535    };
536
537    /// **Routing must be INDEPENDENT of the trajectory hint.**
538    ///
539    /// This test previously asserted the opposite: that the hint reached the bandit's key. It was
540    /// written to catch a wiring bug, and it did. The hint has since been measured three times
541    /// (ADR 0012) — MBPP killed it at −3.4%, agentic SWE-bench killed it on the quality leg, and a
542    /// third run abstained — so the acting logic is gone and this asserts its absence.
543    ///
544    /// Two requests differing ONLY in trajectory difficulty must now share an arm, which is what
545    /// makes the bandit's learned statistics dense again. The hint is still extracted and recorded
546    /// in the audit trace: it costs nothing, it is informative, and a future workload with
547    /// expensive retries may yet justify acting on it. What is withdrawn is the claim that acting
548    /// on it pays.
549    #[test]
550    fn the_trajectory_hint_does_not_affect_routing() {
551        let mut easy = Features::new(TaskKind::CodeEdit);
552        easy.prompt_token_bucket = 6;
553        let mut hard = easy.clone();
554        hard.difficulty_hint = 3;
555
556        assert_eq!(
557            ContextBucket::from_features(&easy),
558            ContextBucket::from_features(&hard),
559            "the hint is refuted as a routing signal — requests differing only in it must share an \
560             arm, or the bandit is re-fragmented by a dimension that measured negative"
561        );
562        // ...and the hint is still present on the feature vector, for the trace and for future study.
563        assert_eq!(hard.difficulty_hint, 3);
564    }
565
566    fn ctx_code() -> ContextBucket {
567        ContextBucket {
568            task_kind: TaskKind::CodeEdit,
569            prompt_bucket_coarse: 3,
570        }
571    }
572
573    fn ctx_chat() -> ContextBucket {
574        ContextBucket {
575            task_kind: TaskKind::Chat,
576            prompt_bucket_coarse: 3,
577        }
578    }
579
580    const HAIKU: &str = "anthropic/claude-haiku-4-5";
581    const SONNET: &str = "anthropic/claude-sonnet-5";
582
583    #[test]
584    fn cold_start_returns_rung_0() {
585        let b = StartRungBandit::new(50, 1.0);
586        let prices = PriceTable::defaults();
587        let ladder = vec![HAIKU.to_owned(), SONNET.to_owned()];
588        // No observations at all → cold start.
589        assert_eq!(b.choose_start(&ctx_code(), &ladder, &prices), 0);
590    }
591
592    #[test]
593    fn empty_ladder_returns_rung_0() {
594        let b = StartRungBandit::new(50, 1.0);
595        let prices = PriceTable::defaults();
596        assert_eq!(b.choose_start(&ctx_code(), &[], &prices), 0);
597    }
598
599    #[test]
600    fn after_enough_rung0_fails_rung1_passes_picks_rung1_for_that_context() {
601        let mut b = StartRungBandit::new(50, 1.0);
602        let prices = PriceTable::defaults();
603        let ladder = vec![HAIKU.to_owned(), SONNET.to_owned()];
604
605        // Feed 60 rung-0 fails + 60 rung-1 passes for ctx_code → bandit should skip rung 0.
606        let code = ctx_code();
607        for _ in 0..60 {
608            b.observe(&code, 0, Verdict::Fail);
609            b.observe(&code, 1, Verdict::Pass);
610        }
611        // Expected cost E[0] = price_haiku + (1 - p̂_0) * price_sonnet  >> E[1] = price_sonnet.
612        assert_eq!(
613            b.choose_start(&code, &ladder, &prices),
614            1,
615            "should skip rung 0 after observing it always fails"
616        );
617
618        // A DIFFERENT context with no data stays cold (returns 0), not influenced by code's data.
619        let chat = ctx_chat();
620        assert_eq!(
621            b.choose_start(&chat, &ladder, &prices),
622            0,
623            "different context must be independent (cold start)"
624        );
625    }
626
627    #[test]
628    fn abstain_verdicts_are_not_counted() {
629        let mut b = StartRungBandit::new(2, 1.0); // tiny min_obs to isolate the logic
630        let code = ctx_code();
631        // Only feed abstains — they must not push us past cold start.
632        for _ in 0..100 {
633            b.observe(&code, 0, Verdict::Abstain);
634        }
635        let prices = PriceTable::defaults();
636        let ladder = vec![HAIKU.to_owned(), SONNET.to_owned()];
637        // N = 0 (abstains excluded) < min_obs=2 → rung 0.
638        assert_eq!(b.choose_start(&code, &ladder, &prices), 0);
639    }
640
641    #[test]
642    fn single_rung_ladder_always_returns_0() {
643        let mut b = StartRungBandit::new(1, 1.0);
644        let code = ctx_code();
645        b.observe(&code, 0, Verdict::Fail);
646        b.observe(&code, 0, Verdict::Pass);
647        let prices = PriceTable::defaults();
648        let ladder = vec![HAIKU.to_owned()];
649        assert_eq!(b.choose_start(&code, &ladder, &prices), 0);
650    }
651
652    /// Build a minimal attempt with a given rung and verdict (for warm-start tests).
653    fn stub_attempt(rung: u32, verdict: Verdict) -> Attempt {
654        Attempt {
655            rung,
656            model: HAIKU.to_owned(),
657            provider: "anthropic".to_owned(),
658            in_tokens: 1000,
659            cache_write_tokens: 0,
660            cache_read_tokens: 0,
661            out_tokens: 500,
662            cost_usd: 0.001,
663            latency_ms: 10,
664            gates: vec![],
665            verdict,
666            truncated: false,
667        }
668    }
669
670    #[test]
671    fn from_features_derives_bucket_correctly() {
672        let mut f = Features::new(TaskKind::CodeEdit);
673        f.prompt_token_bucket = 7; // coarse = 7/2 = 3
674        let b = ContextBucket::from_features(&f);
675        assert_eq!(b.task_kind, TaskKind::CodeEdit);
676        assert_eq!(b.prompt_bucket_coarse, 3);
677    }
678
679    #[test]
680    fn feed_trace_attempts_populates_counts() {
681        let mut bandit = StartRungBandit::new(2, 1.0);
682        let ctx = ctx_code();
683        let attempts = vec![
684            stub_attempt(0, Verdict::Fail),
685            stub_attempt(1, Verdict::Pass),
686        ];
687        bandit.feed_trace_attempts(&ctx, &attempts);
688
689        // Verify internally: 1 fail at rung 0, 1 pass at rung 1 → total N=2 ≥ min_obs=2.
690        // With min_obs=2 and only rung-0 failing, the bandit should prefer rung 1.
691        let prices = PriceTable::defaults();
692        let ladder = vec![HAIKU.to_owned(), SONNET.to_owned()];
693        // N=2, p̂_0 = 0/1 + sqrt(ln2/1) ≈ 0.83, p̂_1 = 1/1 + ... ≥ 1.0
694        // E[0] = price_haiku + (1-0.83)*price_sonnet; E[1] = price_sonnet.
695        // Whether bandit picks 0 or 1 depends on exact math — just verify it doesn't panic.
696        let _ = bandit.choose_start(&ctx, &ladder, &prices);
697    }
698
699    // ---- Guarantee invariant: bandit on + start rung fails → escalates, never serves failing output.
700    // (Integration with the router is tested in router.rs; here we prove the math doesn't
701    // synthesise a "free pass" when an observed rung has 0% pass rate.)
702    #[test]
703    fn zero_pass_rate_rung_is_not_optimistic() {
704        // After many fail observations at rung 0 and pass observations at rung 1,
705        // choosing E[s] should prefer s=1 even with c=0 (no exploration bonus).
706        let mut b = StartRungBandit::new(10, 0.0); // c=0: pure exploitation
707        let ctx = ctx_code();
708        for _ in 0..20 {
709            b.observe(&ctx, 0, Verdict::Fail);
710            b.observe(&ctx, 1, Verdict::Pass);
711        }
712        let prices = PriceTable::defaults();
713        let ladder = vec![HAIKU.to_owned(), SONNET.to_owned()];
714        // p̂_0 = 0 (c=0), p̂_1 = 1.0
715        // E[0] = price_haiku + 1.0*price_sonnet;  E[1] = price_sonnet
716        // E[0] > E[1] → choose start=1.
717        assert_eq!(b.choose_start(&ctx, &ladder, &prices), 1);
718    }
719
720    // ---- Warm-start from disk --------------------------------------------------------
721
722    /// Helper: build a minimal enforce-mode trace with one attempt at rung 0 (pass) and one
723    /// at rung 1 (fail), carrying the given Features so ContextBucket::from_features works.
724    fn trace_with_features_and_attempts(features: Features, attempts: Vec<Attempt>) -> Trace {
725        let mut trace = Trace {
726            trace_id: uuid::Uuid::now_v7(),
727            prev_hash: GENESIS_HASH.to_owned(),
728            tenant_id: "test-tenant".to_owned(),
729            session_id: "sess-warm".to_owned(),
730            ts: jiff::Timestamp::now(),
731            mode: Mode::Enforce,
732            policy: PolicyRef {
733                id: "test@v0".to_owned(),
734                explore: false,
735                propensity: None,
736                mode_profile: None,
737            },
738            request: RequestInfo {
739                api: "anthropic.messages".to_owned(),
740                prompt_hash: "deadbeef".to_owned(),
741                features,
742            },
743            attempts,
744            deferred: vec![],
745            final_: FinalOutcome {
746                served_rung: Some(0),
747                served_from: ServedFrom::Attempt,
748                total_cost_usd: 0.001,
749                gate_cost_usd: 0.0,
750                total_latency_ms: 10,
751                escalations: 0,
752                counterfactual_baseline_usd: 0.001,
753                savings_usd: 0.0,
754                cache_source: None,
755            },
756            probe: None,
757            rollout: None,
758            shadow: None,
759            route_ix: None,
760            predicted_pass: None,
761            decision_prior: None,
762            elastic: None,
763            risk: None,
764            experiment: None,
765        };
766        trace.recompute_savings();
767        trace
768    }
769
770    /// Warm-start test: write traces to a temp store, replay them into a fresh bandit,
771    /// assert the learned counts drive the expected start-rung choice.
772    #[tokio::test]
773    async fn warm_start_from_trace_store_replays_counts() {
774        let db = std::env::temp_dir().join(format!("bandit-warmstart-{}.db", uuid::Uuid::now_v7()));
775        let (tx, writer) = crate::store::open(&db).unwrap();
776
777        // Build features that land in ctx_code (TaskKind::CodeEdit, bucket_coarse=3).
778        let mut f = Features::new(TaskKind::CodeEdit);
779        f.prompt_token_bucket = 7; // coarse = 7/2 = 3
780
781        // 60 traces: rung 0 always fails, rung 1 always passes.
782        for _ in 0..60 {
783            let attempts = vec![
784                stub_attempt(0, Verdict::Fail),
785                stub_attempt(1, Verdict::Pass),
786            ];
787            tx.try_send(trace_with_features_and_attempts(f.clone(), attempts))
788                .unwrap();
789        }
790        drop(tx);
791        writer.await.unwrap();
792
793        // Replay the stored traces into a fresh bandit (mirrors run.rs warm-start logic).
794        let mut bandit = StartRungBandit::new(50, 0.0); // c=0: pure exploitation
795        let stored = crate::store::load_all_traces(&db).unwrap();
796        assert_eq!(stored.len(), 60, "all traces must be stored");
797        for trace in &stored {
798            let ctx = ContextBucket::from_features(&trace.request.features);
799            bandit.feed_trace_attempts(&ctx, &trace.attempts);
800        }
801
802        // After warm-start: 60 rung-0 fails + 60 rung-1 passes → bandit prefers rung 1.
803        let prices = PriceTable::defaults();
804        let ladder = vec![HAIKU.to_owned(), SONNET.to_owned()];
805        let ctx = ContextBucket::from_features(&f);
806        assert_eq!(
807            bandit.choose_start(&ctx, &ladder, &prices),
808            1,
809            "warm-started bandit must prefer rung 1 after 60 fail/pass pairs"
810        );
811
812        // A context with different features sees no data — cold start, returns 0.
813        let mut f2 = Features::new(TaskKind::Chat);
814        f2.prompt_token_bucket = 7;
815        let ctx2 = ContextBucket::from_features(&f2);
816        assert_eq!(
817            bandit.choose_start(&ctx2, &ladder, &prices),
818            0,
819            "different context must be independent"
820        );
821
822        let _ = std::fs::remove_file(&db);
823    }
824
825    // ---- Verified predictive routing: choose_start_with_prior --------------------------
826
827    #[test]
828    fn prior_none_matches_choose_start_with_propensity() {
829        // Property check: with no prior, the two methods must make identical decisions given
830        // identical bandit state and PRNG state (same seed, same call sequence).
831        let ladder = vec![HAIKU.to_owned(), SONNET.to_owned()];
832        let prices = PriceTable::defaults();
833        let ctx = ctx_code();
834
835        let mut a = StartRungBandit::with_algorithm(10, 1.0, Algorithm::Thompson, 1.0, 123);
836        let mut b = StartRungBandit::with_algorithm(10, 1.0, Algorithm::Thompson, 1.0, 123);
837        for _ in 0..30 {
838            a.observe(&ctx, 0, Verdict::Fail);
839            a.observe(&ctx, 1, Verdict::Pass);
840            b.observe(&ctx, 0, Verdict::Fail);
841            b.observe(&ctx, 1, Verdict::Pass);
842        }
843
844        for _ in 0..20 {
845            let want = a.choose_start_with_propensity(&ctx, &ladder, &prices);
846            let got = b.choose_start_with_prior(&ctx, &ladder, &prices, None);
847            assert_eq!(
848                want, got,
849                "prior=None must be byte-identical to the existing method"
850            );
851        }
852    }
853
854    #[test]
855    fn confident_hard_prior_starts_above_zero_with_no_observations() {
856        // n=0 (no bandit warm-start at all): a confident prior that rung 0/1 are hopeless and
857        // rung 2 is where this request will pass must drive the start rung above 0, with no
858        // min_observations cliff blocking it.
859        let ladder = vec![
860            HAIKU.to_owned(),
861            SONNET.to_owned(),
862            "anthropic/claude-opus-4-8".to_owned(),
863        ];
864        let prices = PriceTable::defaults();
865        let ctx = ctx_code();
866        let hard_prior = firstpass_core::cumulative_pass(&[0.05, 0.05, 0.9]).unwrap();
867
868        let mut b = StartRungBandit::with_algorithm(50, 1.0, Algorithm::Ucb1, 1.0, 1)
869            .with_prior_strength(10.0);
870        let (choice, _) = b.choose_start_with_prior(&ctx, &ladder, &prices, Some(&hard_prior));
871        assert!(
872            choice > 0,
873            "a confident hard prior must start above rung 0 with zero observations, got {choice}"
874        );
875    }
876
877    #[test]
878    fn confident_easy_prior_starts_at_zero_with_no_observations() {
879        let ladder = vec![
880            HAIKU.to_owned(),
881            SONNET.to_owned(),
882            "anthropic/claude-opus-4-8".to_owned(),
883        ];
884        let prices = PriceTable::defaults();
885        let ctx = ctx_code();
886        let easy_prior = firstpass_core::cumulative_pass(&[0.95, 0.03, 0.02]).unwrap();
887
888        let mut b = StartRungBandit::with_algorithm(50, 1.0, Algorithm::Ucb1, 1.0, 1)
889            .with_prior_strength(10.0);
890        let (choice, _) = b.choose_start_with_prior(&ctx, &ladder, &prices, Some(&easy_prior));
891        assert_eq!(
892            choice, 0,
893            "a confident easy prior must start at rung 0 with zero observations, got {choice}"
894        );
895    }
896
897    #[test]
898    fn many_observations_override_a_wrong_prior() {
899        // The prior confidently claims rung 0 is hopeless, but 200 real observations say
900        // otherwise — with strength=10 (pseudo-count 10 vs 200 real), data must win.
901        let ladder = vec![HAIKU.to_owned(), SONNET.to_owned()];
902        let prices = PriceTable::defaults();
903        let ctx = ctx_code();
904        let wrong_prior = firstpass_core::cumulative_pass(&[0.02, 0.98]).unwrap();
905
906        let mut b = StartRungBandit::with_algorithm(50, 0.0, Algorithm::Ucb1, 1.0, 1)
907            .with_prior_strength(10.0);
908        for _ in 0..200 {
909            b.observe(&ctx, 0, Verdict::Pass);
910        }
911        let (choice, _) = b.choose_start_with_prior(&ctx, &ladder, &prices, Some(&wrong_prior));
912        assert_eq!(
913            choice, 0,
914            "200 real passes at rung 0 must override a wrong low-strength prior, got {choice}"
915        );
916    }
917
918    #[test]
919    fn mismatched_prior_length_falls_back_to_propensity_method() {
920        let ladder = vec![HAIKU.to_owned(), SONNET.to_owned()];
921        let prices = PriceTable::defaults();
922        let ctx = ctx_code();
923        let mut a = StartRungBandit::new(50, 1.0);
924        let mut b = StartRungBandit::new(50, 1.0);
925        let bad_prior = vec![0.5]; // wrong length for a 2-rung ladder
926        assert_eq!(
927            a.choose_start_with_propensity(&ctx, &ladder, &prices),
928            b.choose_start_with_prior(&ctx, &ladder, &prices, Some(&bad_prior)),
929        );
930    }
931
932    #[test]
933    fn beta_sampler_mean_matches_posterior_mean() {
934        let mut b = StartRungBandit::with_algorithm(0, 1.0, Algorithm::Thompson, 1.0, 0xDEADBEEF);
935        // Beta(8, 2): mean 0.8. 4000 draws give a tight empirical mean.
936        let ctx = ctx_code();
937        for _ in 0..7 {
938            b.observe(&ctx, 0, Verdict::Pass);
939        }
940        b.observe(&ctx, 0, Verdict::Fail);
941        // counts: pass 7, fail 1 → Beta(8, 2), mean 0.8.
942        let mean: f64 = (0..4000).map(|_| b.thompson_pass(&ctx, 0)).sum::<f64>() / 4000.0;
943        assert!(
944            (mean - 0.8).abs() < 0.03,
945            "Beta(8,2) sample mean should be ~0.8, got {mean}"
946        );
947        // Unobserved arm = Beta(1,1) uniform, mean 0.5.
948        let mean_u: f64 = (0..4000).map(|_| b.thompson_pass(&ctx, 5)).sum::<f64>() / 4000.0;
949        assert!(
950            (mean_u - 0.5).abs() < 0.03,
951            "uniform prior mean ~0.5, got {mean_u}"
952        );
953    }
954
955    #[test]
956    fn thompson_converges_to_skipping_a_hopeless_cheap_rung() {
957        let mut b = StartRungBandit::with_algorithm(10, 1.0, Algorithm::Thompson, 1.0, 42);
958        let ctx = ctx_code();
959        let ladder = vec![HAIKU.to_owned(), SONNET.to_owned()];
960        let prices = PriceTable::defaults();
961        // Rung 0 always fails, rung 1 always passes → starting at 1 is cheaper in expectation.
962        for _ in 0..80 {
963            b.observe(&ctx, 0, Verdict::Fail);
964            b.observe(&ctx, 1, Verdict::Pass);
965        }
966        let picks_rung1 = (0..100)
967            .filter(|_| b.choose_start_with_propensity(&ctx, &ladder, &prices).0 == 1)
968            .count();
969        assert!(
970            picks_rung1 >= 90,
971            "TS should overwhelmingly skip the hopeless cheap rung, picked rung 1 {picks_rung1}/100"
972        );
973    }
974
975    #[test]
976    fn discounting_adapts_after_a_distribution_flip() {
977        let ctx = ctx_code();
978        // History: rung 0 failed 200 times. Then the world flips: 40 recent passes.
979        let feed = |b: &mut StartRungBandit| {
980            for _ in 0..200 {
981                b.observe(&ctx, 0, Verdict::Fail);
982            }
983            for _ in 0..40 {
984                b.observe(&ctx, 0, Verdict::Pass);
985            }
986        };
987        let mut discounted = StartRungBandit::with_algorithm(10, 1.0, Algorithm::Thompson, 0.95, 7);
988        let mut undiscounted =
989            StartRungBandit::with_algorithm(10, 1.0, Algorithm::Thompson, 1.0, 7);
990        feed(&mut discounted);
991        feed(&mut undiscounted);
992        let p_disc = discounted.pass_estimate(&ctx, 0).unwrap();
993        let p_flat = undiscounted.pass_estimate(&ctx, 0).unwrap();
994        assert!(
995            p_disc > 0.7 && p_flat < 0.25,
996            "discounted tracks the flip (got {p_disc:.2}), undiscounted lags (got {p_flat:.2})"
997        );
998    }
999
1000    #[test]
1001    fn thompson_propensity_matches_empirical_selection_frequency() {
1002        let mut b = StartRungBandit::with_algorithm(10, 1.0, Algorithm::Thompson, 1.0, 99);
1003        let ctx = ctx_code();
1004        let ladder = vec![HAIKU.to_owned(), SONNET.to_owned()];
1005        let prices = PriceTable::defaults();
1006        // Ambiguous data: rung 0 passes ~half the time → genuinely stochastic selection.
1007        for _ in 0..20 {
1008            b.observe(&ctx, 0, Verdict::Pass);
1009            b.observe(&ctx, 0, Verdict::Fail);
1010            b.observe(&ctx, 1, Verdict::Pass);
1011        }
1012        // Empirical frequency of each choice over many decisions vs the logged propensity.
1013        let mut freq = std::collections::HashMap::new();
1014        let mut props: Vec<(u32, f64)> = Vec::new();
1015        for _ in 0..400 {
1016            let (c, p) = b.choose_start_with_propensity(&ctx, &ladder, &prices);
1017            *freq.entry(c).or_insert(0u32) += 1;
1018            props.push((c, p.expect("thompson always logs a propensity")));
1019        }
1020        for (arm, count) in freq {
1021            let empirical = f64::from(count) / 400.0;
1022            let mean_logged: f64 = {
1023                let logged: Vec<f64> = props
1024                    .iter()
1025                    .filter(|(c, _)| *c == arm)
1026                    .map(|(_, p)| *p)
1027                    .collect();
1028                logged.iter().sum::<f64>() / logged.len() as f64
1029            };
1030            assert!(
1031                (empirical - mean_logged).abs() < 0.15,
1032                "arm {arm}: empirical {empirical:.2} vs logged propensity {mean_logged:.2}"
1033            );
1034        }
1035    }
1036
1037    #[test]
1038    fn pass_estimate_cold_and_warm() {
1039        let mut b = StartRungBandit::with_algorithm(10, 1.0, Algorithm::Thompson, 1.0, 5);
1040        let ctx = ctx_code();
1041        assert!(
1042            b.pass_estimate(&ctx, 0).is_none(),
1043            "cold context: no estimate"
1044        );
1045        for _ in 0..12 {
1046            b.observe(&ctx, 0, Verdict::Pass);
1047        }
1048        let p = b.pass_estimate(&ctx, 0).expect("warm");
1049        assert!(p > 0.85, "12/12 passes: posterior mean high, got {p}");
1050        assert!(
1051            b.pass_estimate(&ctx, 3).is_none(),
1052            "unobserved arm: no estimate"
1053        );
1054    }
1055
1056    /// The prompt-size band must invert to a monotonically increasing token estimate, since the
1057    /// whole point is that a bigger request implies a costlier cheap attempt.
1058    #[test]
1059    fn representative_tokens_grow_with_the_prompt_band() {
1060        let sizes: Vec<u64> = (0..5)
1061            .map(|b| {
1062                ContextBucket {
1063                    task_kind: TaskKind::CodeEdit,
1064                    prompt_bucket_coarse: b,
1065                }
1066                .representative_prompt_tokens()
1067            })
1068            .collect();
1069        assert!(
1070            sizes.windows(2).all(|w| w[1] > w[0]),
1071            "estimate must be strictly increasing in the band, got {sizes:?}"
1072        );
1073        // And bounded, so a pathological bucket cannot price a request absurdly.
1074        let huge = ContextBucket {
1075            task_kind: TaskKind::CodeEdit,
1076            prompt_bucket_coarse: u32::MAX,
1077        };
1078        assert!(huge.representative_prompt_tokens() <= 1 << 20);
1079    }
1080
1081    /// The behaviour the fixed nominal token count made impossible.
1082    ///
1083    /// Two contexts with **identical** gate statistics but different prompt sizes must be allowed
1084    /// to reach different start rungs, because `start cheap ⟺ c₀ < p·c₁` scales both sides with
1085    /// request size. Under the old constant-token pricing the ratio `c₀/c₁` was the same for every
1086    /// request, so this decision could only ever depend on `p` — and the measured 2.16× adverse
1087    /// selection was invisible to it.
1088    ///
1089    /// A cheap rung that is *not* meaningfully cheaper than the top rung should not be attempted
1090    /// when its pass rate is poor; here the ladder is deliberately shallow so that the marginal
1091    /// call is decided by price, which is exactly the regime this fix targets.
1092    #[test]
1093    fn start_rung_can_depend_on_request_size_not_just_pass_rate() {
1094        let prices = PriceTable::defaults();
1095        let ladder = vec![HAIKU.to_owned(), SONNET.to_owned()];
1096
1097        // Identical statistics, applied to a small and a large prompt band.
1098        let mut small = StartRungBandit::new(4, 0.0);
1099        let mut large = StartRungBandit::new(4, 0.0);
1100        let c_small = ContextBucket {
1101            task_kind: TaskKind::CodeEdit,
1102            prompt_bucket_coarse: 1,
1103        };
1104        let c_large = ContextBucket {
1105            task_kind: TaskKind::CodeEdit,
1106            prompt_bucket_coarse: 8,
1107        };
1108        for _ in 0..10 {
1109            small.observe(&c_small, 0, Verdict::Fail);
1110            small.observe(&c_small, 1, Verdict::Pass);
1111            large.observe(&c_large, 0, Verdict::Fail);
1112            large.observe(&c_large, 1, Verdict::Pass);
1113        }
1114
1115        // Both should skip the hopeless cheap rung; the point of the test is that the decision is
1116        // now computed from a size-scaled price rather than a constant, so the two contexts price
1117        // the same ladder differently.
1118        assert_eq!(small.choose_start(&c_small, &ladder, &prices), 1);
1119        assert_eq!(large.choose_start(&c_large, &ladder, &prices), 1);
1120        assert!(
1121            c_large.representative_prompt_tokens() > c_small.representative_prompt_tokens() * 8,
1122            "the large context must price the same ladder materially higher"
1123        );
1124    }
1125}