cratefield_core/budget.rs
1//! Outbound HTTP budgets (issue #765): a per-upstream (and per-API-key)
2//! token bucket shared across isolates, a per-provider daily quota, and a
3//! `Retry-After`-aware retry policy — one [`HttpClient`] decorator an
4//! adapter opts into by wrapping the client it already holds.
5//!
6//! An outbound request spends three budgets, in the order they can refuse
7//! it:
8//!
9//! 1. **Pacing.** A [`Budget`] names a token bucket — `capacity` tokens of
10//! burst, refilled at `refill_per_sec` — and what one request costs. The
11//! bucket lives in the venture database, one row per key, so every
12//! isolate of every Worker instance spends the *same* bucket: admission
13//! and the refill-and-spend are **one statement** (the same guarded
14//! upsert shape [`Usage`] is built on, issue #588), so concurrent
15//! callers cannot both spend the last token. A shortfall that refills
16//! within [`RetryPolicy::max_delay`] is waited out in-process — that is
17//! pacing, not refusal, and it is bounded by `MAX_PACING_WAITS` — and
18//! anything longer comes back as [`HttpError::BudgetExhausted`] with the
19//! wait as `retry_after`.
20//! 2. **Daily quota.** `Budget::daily_limit` caps requests per UTC day
21//! through [`Usage`] (`Period::Day`), so "10 000 calls a day" is a
22//! durable allowance, not a hope. Crossing the limit — or its 80%
23//! warning mark — emits [`EVENT_QUOTA_EXHAUSTED`] /
24//! [`EVENT_QUOTA_WARNING`] when [`BudgetedHttpClient::with_events`] is
25//! set. The alerts are at-most-once **by claim, not by arithmetic**:
26//! the increment is atomic but the read-back each caller sees is a
27//! separate statement, so concurrent callers can all observe the same
28//! crossing. What keeps the emission to one is a durable claim row —
29//! the first caller whose guarded insert reports one affected row owns
30//! the emission, every later observer's reports none and skips (the
31//! table: [`BudgetedHttpClient::create_alert_table_sql`]; the claim is
32//! keyed per UTC day, so a new day alerts afresh). The same rule trades
33//! the old exact-match arithmetic for an inequality: a burst that leaps
34//! the warning mark in one step — any limit of 4 or fewer, where
35//! `ceil(limit * 4 / 5)` is the limit itself — emits only
36//! [`EVENT_QUOTA_EXHAUSTED`]. A budget with `daily_limit: None` is
37//! untracked and pays no database round-trip.
38//! 3. **Retries.** A 429/503 or a transport failure on an **idempotent**
39//! request (safe methods by default, a request-inserted [`Idempotent`]
40//! extension to override) is retried with exponential backoff, honouring
41//! the upstream's own `Retry-After` when it states one — both of its
42//! forms, through the one parser (issue #214). Delays clamp to
43//! `max_delay` each and to `max_total` for the whole exchange; a retry
44//! that no longer fits is not taken, and the last response or error
45//! passes through as-is. Bodies are never consumed to make this
46//! decision. Retries never re-pay the budget: pacing and the daily
47//! quota are spent once, before the first attempt, and the retries
48//! re-send through the inner client without touching either — one
49//! budget unit per logical call, across at most `max_retries + 1`
50//! upstream hits. Deliberate: the quota counts calls, not wire
51//! attempts.
52//!
53//! # The worst case, added up
54//!
55//! The budgets stack, and so do their waits: one `send()` may spend
56//! `MAX_PACING_WAITS * max_delay` pacing (4 × 5 s = 20 s on the default
57//! [`RetryPolicy`]), then up to `max_total` more in the retry exchange
58//! (15 s), then whatever the inner client's own response timeout allows —
59//! up to the 30 s `MAX_RESPONSE_TIMEOUT` ceiling `ports/http.rs` caps it
60//! at. On the defaults that sum is 65 s, past the 30 s request bound a
61//! Worker runs under, so a caller with a strict deadline lowers
62//! [`RetryPolicy`] (and its router's numbers); the decorator does not
63//! shorten the sum for it.
64//!
65//! Anything the budget layer does not govern — unrouted requests,
66//! non-idempotent failures, other error variants — reaches the inner client
67//! unchanged, exactly as
68//! [`BoundedHttpClient`](crate::ports::BoundedHttpClient) passes through
69//! what it does not bound.
70//!
71//! # Wiring
72//!
73//! ```ignore
74//! let client = BudgetedHttpClient::new(inner, clock, db, router)
75//! .with_retry(RetryPolicy::default().with_max_retries(3))
76//! .with_events(ctx.events.clone())
77//! .with_defer(ctx.ports.defer.clone()?);
78//! ```
79//!
80//! The decorator needs three tables, and the venture ships all three as
81//! migrations: [`BudgetedHttpClient::create_bucket_table_sql`] for the
82//! pacing buckets, [`Usage::create_table_sql`] for the daily quota, and
83//! [`BudgetedHttpClient::create_alert_table_sql`] for the quota-alert
84//! claims (default names [`DEFAULT_BUCKET_TABLE`], [`DEFAULT_QUOTA_TABLE`]
85//! and [`DEFAULT_ALERT_TABLE`], overridable with `with_bucket_table` /
86//! `with_quota_table` / `with_alert_table`). Only the alert path writes
87//! the claim table, and it creates the table on demand if a claim ever
88//! meets its absence — the migration is the contract, the auto-create the
89//! safety net that keeps the alerts strict instead of dropping them. A
90//! client with `with_events` unset never touches the claim table at all.
91//!
92//! Tokens are `f64` end to end — capacity, refill, cost — so a fractional
93//! refill (0.5 tokens a second, a 2.5-token cost) needs no scaling. The
94//! *read* is wider than the write: D1's JSON bridge hands a safe integral
95//! JS number back as an integer, not a double, so the advisory read falls
96//! back from the float shapes to the integral ones (`tokens_of` below) —
97//! a bucket parked at exactly `9.0` must read as `9.0`, not as a fresh
98//! bucket. All the arithmetic is pure functions below, unit-tested without
99//! a database.
100
101use std::future::Future;
102use std::pin::Pin;
103use std::sync::Arc;
104use std::time::Duration;
105
106use async_trait::async_trait;
107use bytes::Bytes;
108use serde_json::json;
109
110use crate::events::EventBus;
111use crate::ports::{
112 ByteStream, Clock, Database, Defer, HttpClient, HttpError, IdGen, NoopDefer, Row, Statement,
113 UlidIdGen,
114};
115use crate::retry_after;
116use crate::scope::Scope;
117use crate::usage::{Consumption, Period, Usage, rfc3339};
118
119/// Emitted when a key's daily quota is crossed — the first budgeted send
120/// whose read-back observes `used >= limit` and wins the day's durable
121/// claim. Payload: `{"subject": <key>, "used": <used>, "limit": <limit>}`,
122/// where `used` is the observed total and can pass `limit`. At most once
123/// per key per UTC day: the claim row is keyed `(subject, period, kind)`
124/// and the day is part of the key, so a new day can alert again.
125pub const EVENT_QUOTA_EXHAUSTED: &str = "budget.quota_exhausted";
126
127/// Emitted when a key lands past the daily quota's 80% warning mark
128/// (`ceil(limit * 4 / 5)`) — the same durable claim under a different
129/// `kind`, so the two never suppress each other within a day. The send
130/// that crosses the limit in the same step emits [`EVENT_QUOTA_EXHAUSTED`]
131/// instead, which is what happens for every limit of 4 or fewer (where the
132/// mark *is* the limit) and for any burst that leaps the mark in one
133/// increment.
134pub const EVENT_QUOTA_WARNING: &str = "budget.quota_warning";
135
136/// The meter name every daily-quota spend is counted under, so
137/// [`Usage::history`] reads one meter across all budgeted keys.
138const QUOTA_METER: &str = "http-requests";
139
140/// The default pacing-bucket table [`BudgetedHttpClient`] spends from;
141/// override with `with_bucket_table` when a venture renames it.
142pub const DEFAULT_BUCKET_TABLE: &str = "http_bucket";
143
144/// The default daily-quota table [`BudgetedHttpClient`] meters into;
145/// override with `with_quota_table` when a venture renames it.
146pub const DEFAULT_QUOTA_TABLE: &str = "http_usage";
147
148/// The default quota-alert claim table [`BudgetedHttpClient`] gates its
149/// [`EVENT_QUOTA_WARNING`] / [`EVENT_QUOTA_EXHAUSTED`] emissions with;
150/// override with `with_alert_table` when a venture renames it.
151pub const DEFAULT_ALERT_TABLE: &str = "budget_alerts";
152
153/// How many times one send may wait out a token-bucket shortfall before
154/// giving up and reporting [`HttpError::BudgetExhausted`]. Each wait is the
155/// computed refill time, itself capped by [`RetryPolicy::max_delay`], so
156/// pacing gives up after at most `MAX_PACING_WAITS * max_delay` — the bound
157/// that keeps a permanently dry bucket from holding a request forever.
158const MAX_PACING_WAITS: u32 = 4;
159
160/// Fraction of the daily quota the warning event marks, as a numerator over
161/// its denominator (`4 / 5 == 80%`), kept integral so the threshold is
162/// exact.
163const QUOTA_WARNING_NUMERATOR: u64 = 4;
164const QUOTA_WARNING_DENOMINATOR: u64 = 5;
165
166/// Request extension overriding the default method-based idempotency
167/// classification. Safe methods (`GET`, `HEAD`, `OPTIONS`) are retried on a
168/// 429/503 or a transport failure; a caller that knows better — a `POST`
169/// the upstream de-duplicates by idempotency key, or a `GET` with side
170/// effects — pins the answer with this extension, which never reaches the
171/// wire.
172#[derive(Debug, Clone, Copy, PartialEq, Eq)]
173pub struct Idempotent(pub bool);
174
175/// The budget a single request spends against, as the router names it.
176#[derive(Debug, Clone, PartialEq)]
177pub struct Budget {
178 /// Identifies the upstream (and the API key within it), e.g.
179 /// `"dexscreener"` or `"rpc:3f9a…"`. The token-bucket row and the daily
180 /// quota counter are keyed by exactly this string.
181 pub key: String,
182 /// Token-bucket size: the burst the key may spend at once, and the
183 /// level every bucket refills toward.
184 pub capacity: f64,
185 /// Sustained refill rate, tokens per second.
186 pub refill_per_sec: f64,
187 /// Tokens this one request spends. `1.0` is a plain request; heavier
188 /// endpoints can charge more. A cost above `capacity` can never be
189 /// served and reports exhausted with the wait until a full bucket
190 /// would cover it.
191 pub cost: f64,
192 /// Per-UTC-day request quota, counted through [`Usage`] with
193 /// [`Period::Day`]. `None` is untracked: no quota, no database write.
194 pub daily_limit: Option<u64>,
195}
196
197/// Routes a request to the budget it spends, or `None` to pass it through
198/// un-budgeted. The key should name the upstream and, where credentials
199/// rotate, the key they ride on — never the credential itself.
200pub type BudgetRouter = Arc<dyn Fn(&http::Request<Bytes>) -> Option<Budget> + Send + Sync>;
201
202/// How a failed exchange is retried: `max_retries` extra attempts at most,
203/// exponential backoff from `base` (200 ms, 400 ms, 800 ms, …) clamped to
204/// `max_delay` per wait and `max_total` across the whole exchange. An
205/// upstream-stated `Retry-After` replaces the computed delay and clamps
206/// only to what `max_total` still allows.
207#[derive(Debug, Clone, Copy, PartialEq, Eq)]
208pub struct RetryPolicy {
209 max_retries: u32,
210 base: Duration,
211 max_delay: Duration,
212 max_total: Duration,
213}
214
215impl Default for RetryPolicy {
216 fn default() -> Self {
217 Self {
218 max_retries: 2,
219 base: Duration::from_millis(200),
220 max_delay: Duration::from_secs(5),
221 max_total: Duration::from_secs(15),
222 }
223 }
224}
225
226impl RetryPolicy {
227 /// Extra attempts after the first send.
228 #[must_use]
229 pub fn max_retries(self) -> u32 {
230 self.max_retries
231 }
232
233 /// First computed backoff, doubled every attempt.
234 #[must_use]
235 pub fn base(self) -> Duration {
236 self.base
237 }
238
239 /// Ceiling on one computed wait. An upstream-stated `Retry-After` may
240 /// exceed this; only `max_total` caps it.
241 #[must_use]
242 pub fn max_delay(self) -> Duration {
243 self.max_delay
244 }
245
246 /// Ceiling on everything the retries may wait, added up.
247 #[must_use]
248 pub fn max_total(self) -> Duration {
249 self.max_total
250 }
251
252 /// Sets the retry count.
253 #[must_use]
254 pub fn with_max_retries(mut self, max_retries: u32) -> Self {
255 self.max_retries = max_retries;
256 self
257 }
258
259 /// Sets the first computed backoff.
260 #[must_use]
261 pub fn with_base(mut self, base: Duration) -> Self {
262 self.base = base;
263 self
264 }
265
266 /// Sets the per-wait ceiling.
267 #[must_use]
268 pub fn with_max_delay(mut self, max_delay: Duration) -> Self {
269 self.max_delay = max_delay;
270 self
271 }
272
273 /// Sets the across-the-exchange ceiling.
274 #[must_use]
275 pub fn with_max_total(mut self, max_total: Duration) -> Self {
276 self.max_total = max_total;
277 self
278 }
279}
280
281/// The async sleep pacing and backoff wait through. The default is a real
282/// timer (`futures-timer`, no executor dependency). Runtimes with a native
283/// timer of their own — Workers' `setTimeout`, a test's instant no-op —
284/// install it here with [`BudgetedHttpClient::with_sleeper`]; on a wasm
285/// target, whose std has no timer to borrow, a runtime sleeper is the
286/// supported path (see `default_sleeper`).
287pub type Sleeper = Arc<dyn Fn(Duration) -> Pin<Box<dyn Future<Output = ()> + Send>> + Send + Sync>;
288
289/// Wraps any [`HttpClient`] with the outbound budgets of issue #765:
290/// a shared token bucket per upstream key, a daily quota with alerts, and
291/// `Retry-After`-aware retries for idempotent calls. Adapters opt in by
292/// wrapping the client they already hold — the runtime's
293/// [`BoundedHttpClient`](crate::ports::BoundedHttpClient) stays the
294/// innermost hop, so the port's own bounds
295/// still apply underneath.
296///
297/// The streaming twin ([`HttpClient::send_streaming`]) spends the same
298/// budget the same way before the head, then hands the body through
299/// untouched: the retry loop covers the head only, because a body that
300/// has started arriving can never be replayed.
301pub struct BudgetedHttpClient {
302 inner: Arc<dyn HttpClient>,
303 clock: Arc<dyn Clock>,
304 db: Arc<dyn Database>,
305 router: BudgetRouter,
306 retry: RetryPolicy,
307 events: Option<EventBus>,
308 defer: Arc<dyn Defer>,
309 sleeper: Sleeper,
310 bucket_table: String,
311 quota_table: String,
312 alert_table: String,
313}
314
315impl BudgetedHttpClient {
316 /// Budgets `inner`'s sends through `router`, reading wall time from
317 /// `clock` and spending buckets and quotas against `db`.
318 #[must_use]
319 pub fn new(
320 inner: Arc<dyn HttpClient>,
321 clock: Arc<dyn Clock>,
322 db: Arc<dyn Database>,
323 router: BudgetRouter,
324 ) -> Self {
325 Self {
326 inner,
327 clock,
328 db,
329 router,
330 retry: RetryPolicy::default(),
331 events: None,
332 defer: Arc::new(NoopDefer),
333 sleeper: default_sleeper(),
334 bucket_table: DEFAULT_BUCKET_TABLE.to_owned(),
335 quota_table: DEFAULT_QUOTA_TABLE.to_owned(),
336 alert_table: DEFAULT_ALERT_TABLE.to_owned(),
337 }
338 }
339
340 /// Replaces the retry policy.
341 #[must_use]
342 pub fn with_retry(mut self, policy: RetryPolicy) -> Self {
343 self.retry = policy;
344 self
345 }
346
347 /// Emits [`EVENT_QUOTA_WARNING`] / [`EVENT_QUOTA_EXHAUSTED`] to `bus`
348 /// as the daily quotas are crossed.
349 #[must_use]
350 pub fn with_events(mut self, bus: EventBus) -> Self {
351 self.events = Some(bus);
352 self
353 }
354
355 /// Runs the alert events' handlers through this defer. Without it the
356 /// default [`NoopDefer`] drops them (with a warning), so a deployment
357 /// that subscribes to the quota events mounts the port's defer here.
358 #[must_use]
359 pub fn with_defer(mut self, defer: Arc<dyn Defer>) -> Self {
360 self.defer = defer;
361 self
362 }
363
364 /// Replaces the sleep pacing and backoff wait through — the hook a
365 /// runtime with a native timer (or a test that asserts on delays)
366 /// supplies.
367 #[must_use]
368 pub fn with_sleeper(mut self, sleeper: Sleeper) -> Self {
369 self.sleeper = sleeper;
370 self
371 }
372
373 /// Renames the pacing-bucket table.
374 #[must_use]
375 pub fn with_bucket_table(mut self, table: impl Into<String>) -> Self {
376 self.bucket_table = table.into();
377 self
378 }
379
380 /// Renames the daily-quota table.
381 #[must_use]
382 pub fn with_quota_table(mut self, table: impl Into<String>) -> Self {
383 self.quota_table = table.into();
384 self
385 }
386
387 /// Renames the quota-alert claim table.
388 #[must_use]
389 pub fn with_alert_table(mut self, table: impl Into<String>) -> Self {
390 self.alert_table = table.into();
391 self
392 }
393
394 /// The portable DDL for the pacing-bucket table. The daily-quota table
395 /// is [`Usage::create_table_sql`] and the quota-alert claim table is
396 /// [`create_alert_table_sql`](Self::create_alert_table_sql) — a venture
397 /// budgets only when it has created **all three**.
398 #[must_use]
399 pub fn create_bucket_table_sql(table: &str) -> String {
400 format!(
401 "CREATE TABLE IF NOT EXISTS {table} (\n \
402 bucket TEXT NOT NULL,\n \
403 tokens DOUBLE PRECISION NOT NULL,\n \
404 updated_ms BIGINT NOT NULL,\n \
405 PRIMARY KEY (bucket)\n);"
406 )
407 }
408
409 /// The portable DDL for the quota-alert claim table: one row per
410 /// `(subject, day, kind)` emission already made, the at-most-once gate
411 /// behind [`EVENT_QUOTA_WARNING`] / [`EVENT_QUOTA_EXHAUSTED`]. The
412 /// claim's `subject` column names the quota table too, so two clients
413 /// metering different tables never claim each other's alerts. A
414 /// venture that wires `with_events` ships this as a migration beside
415 /// the other two; a claim that meets its absence creates it on demand
416 /// (the same DDL, `IF NOT EXISTS`) rather than dropping the alert.
417 #[must_use]
418 pub fn create_alert_table_sql(table: &str) -> String {
419 format!(
420 "CREATE TABLE IF NOT EXISTS {table} (\n \
421 subject TEXT NOT NULL,\n \
422 period TEXT NOT NULL,\n \
423 kind TEXT NOT NULL,\n \
424 claimed_ms BIGINT NOT NULL,\n \
425 PRIMARY KEY (subject, period, kind)\n);"
426 )
427 }
428
429 /// Spends one request against the key's daily quota. `daily_limit:
430 /// None` is untracked and skips the database entirely; with a limit,
431 /// refusal carries the period's own `retry_after`.
432 async fn spend_daily_quota(&self, budget: &Budget) -> Result<(), HttpError> {
433 let Some(limit) = budget.daily_limit else {
434 return Ok(());
435 };
436 // One clock reading for the spend and its alert alike, so the two
437 // cannot disagree about which UTC day they landed in.
438 let now = self.clock.now();
439 let outcome = Usage::new(&self.quota_table)
440 .consume(
441 &*self.db,
442 &budget.key,
443 QUOTA_METER,
444 Period::Day,
445 now,
446 1,
447 Some(limit),
448 )
449 .await
450 .map_err(|err| store_failed(&budget.key, &err))?;
451 match outcome {
452 Consumption::Consumed(consumed) => {
453 self.emit_quota_alert(&budget.key, consumed.used, limit, now)
454 .await;
455 Ok(())
456 }
457 Consumption::Exhausted(exhausted) => Err(HttpError::BudgetExhausted {
458 what: budget.key.clone(),
459 retry_after: exhausted.retry_after,
460 }),
461 }
462 }
463
464 /// The quota alerts, at-most-once **by claim**: the read-back total
465 /// every concurrent caller sees can be the same number — `Usage`'s
466 /// increment is atomic, its separate read-back is not — so the
467 /// emission is gated on the durable claim row. The first caller whose
468 /// guarded insert reports one affected row owns the emission; the rest
469 /// report none and skip. The condition is read off the observed total
470 /// as an inequality, `used >= limit` then `used >= warning mark` (kept
471 /// as an else-if, so a burst that leaps the mark in one increment
472 /// emits the exhaustion only).
473 ///
474 /// The scope is built here, not taken from a request — a budgeted send
475 /// may serve no request at all — with a fresh traceable id, the same
476 /// shape `module-connections` uses for its scheduled passes.
477 async fn emit_quota_alert(&self, key: &str, used: u64, limit: u64, now: time::OffsetDateTime) {
478 let Some(bus) = self.events.as_ref() else {
479 return;
480 };
481 let (name, kind) = if used >= limit {
482 (EVENT_QUOTA_EXHAUSTED, "exhausted")
483 } else if used >= warning_threshold(limit) {
484 (EVENT_QUOTA_WARNING, "warning")
485 } else {
486 return;
487 };
488 let claimant = format!("{}:{key}", self.quota_table);
489 let period = rfc3339(Period::Day.window_at(now).start);
490 if !self
491 .claim_quota_alert(&claimant, kind, &period, epoch_millis(now))
492 .await
493 {
494 return;
495 }
496 let scope = Scope {
497 request_id: UlidIdGen.ulid(),
498 defer: Arc::clone(&self.defer),
499 span: tracing::info_span!("http.budget", budget = %key, outcome = kind),
500 };
501 bus.emit_in(
502 &scope,
503 name,
504 json!({ "subject": key, "used": used, "limit": limit }),
505 );
506 }
507
508 /// The durable emission claim: one guarded insert —
509 /// [`SendCooldown`]'s conflict-insert arm (a claim is never renewed; a
510 /// new day is a new row, so the update arm has nothing to renew) —
511 /// answering `true` for the one caller that owns the emission.
512 /// `rows-affected == 1` is the whole contract, and it is the only path
513 /// to an emission.
514 ///
515 /// A claim that meets its missing table creates it (the same DDL a
516 /// venture ships as its migration, `IF NOT EXISTS`) and claims once
517 /// more; a claim that still cannot be written drops the alert with a
518 /// warning. An alert never fails the send it observes — at-most-once
519 /// means a lost alert stays lost, never a retried one.
520 async fn claim_quota_alert(
521 &self,
522 claimant: &str,
523 kind: &str,
524 period: &str,
525 now_ms: i64,
526 ) -> bool {
527 let insert = || claim_statement(&self.alert_table, claimant, kind, period, now_ms);
528 match self.db.execute(&insert()).await {
529 Ok(1) => true,
530 Ok(_) => false,
531 Err(_) => {
532 let created = self
533 .db
534 .execute(&Statement::new(BudgetedHttpClient::create_alert_table_sql(
535 &self.alert_table,
536 )))
537 .await
538 .is_ok();
539 if !created {
540 tracing::warn!(
541 budget = %claimant,
542 kind,
543 "quota alert dropped: the claim table is missing and could not be created"
544 );
545 return false;
546 }
547 matches!(self.db.execute(&insert()).await, Ok(1))
548 }
549 }
550 }
551
552 /// Refills and spends the key's token bucket, atomically. A shortfall
553 /// that refills within `max_delay` is waited out (up to
554 /// `MAX_PACING_WAITS`); anything longer refuses.
555 async fn acquire(&self, budget: &Budget) -> Result<(), HttpError> {
556 // The router's numbers are configuration: a negative or NaN value
557 // collapses to zero here instead of poisoning the arithmetic, which
558 // `f64::max` gives for free (NaN.max(0.0) is 0.0).
559 let capacity = budget.capacity.max(0.0);
560 let cost = budget.cost.max(0.0);
561 let refill_per_sec = budget.refill_per_sec.max(0.0);
562 let mut waits_left = MAX_PACING_WAITS;
563 loop {
564 let now_ms = epoch_millis(self.clock.now());
565 let affected = self
566 .db
567 .execute(&acquire_statement(
568 &self.bucket_table,
569 &budget.key,
570 capacity,
571 cost,
572 refill_per_sec,
573 now_ms,
574 ))
575 .await
576 .map_err(|err| store_failed(&budget.key, &err))?;
577 if affected > 0 {
578 return Ok(());
579 }
580 // Advisory read, no transaction: one row's level, to say *when*
581 // the bucket could carry this request. Another isolate may
582 // spend or refill before the answer lands; the number is a
583 // quotation, not a promise.
584 let (stored, updated_ms) = self.bucket_row(&budget.key, capacity, now_ms).await?;
585 let refilled = refilled_tokens(stored, updated_ms, now_ms, refill_per_sec, capacity);
586 let retry_in = pacing_retry_in(refilled, cost, refill_per_sec);
587 if retry_in > self.retry.max_delay() || waits_left == 0 {
588 return Err(HttpError::BudgetExhausted {
589 what: budget.key.clone(),
590 retry_after: retry_in,
591 });
592 }
593 (self.sleeper)(retry_in).await;
594 waits_left -= 1;
595 }
596 }
597
598 /// The bucket row's `(tokens, updated_ms)`, or a full fresh bucket when
599 /// the key has no row yet (its absence after a failed spend means the
600 /// seed guard refused: `capacity < cost`).
601 async fn bucket_row(
602 &self,
603 key: &str,
604 capacity: f64,
605 now_ms: i64,
606 ) -> Result<(f64, i64), HttpError> {
607 let rows = self
608 .db
609 .query(&Statement::with_values(
610 bucket_read_sql(&self.bucket_table),
611 vec![key.to_owned().into()],
612 ))
613 .await
614 .map_err(|err| store_failed(key, &err))?;
615 Ok(rows
616 .first()
617 .and_then(|row| Some((tokens_of(row)?, row.get::<i64>("updated_ms")?)))
618 .unwrap_or((capacity, now_ms)))
619 }
620
621 /// The send-and-retry loop: pacing and the quota have already admitted
622 /// this request, so what is left is the exchange itself. `send_one`
623 /// produces one attempt's future — the buffered `send` retries the
624 /// whole exchange, `send_streaming` retries only the **head** (a body
625 /// that has started arriving can never be replayed, so once a
626 /// streamed response is in hand it is returned as-is). Each attempt
627 /// clones what it consumes — `Bytes` clones are refcounted, and the
628 /// extensions (the [`Idempotent`] marker included) ride along.
629 async fn exchange_with<B, F, Fut>(
630 &self,
631 idempotent: bool,
632 send_one: F,
633 ) -> Result<http::Response<B>, HttpError>
634 where
635 F: Fn() -> Fut,
636 Fut: Future<Output = Result<http::Response<B>, HttpError>>,
637 {
638 let policy = self.retry;
639 let mut waited = Duration::ZERO;
640 let mut retries_left = policy.max_retries();
641 let mut attempt = 0_u32;
642 loop {
643 let outcome = send_one().await;
644 match outcome {
645 Ok(response) => {
646 let throttled = matches!(response.status().as_u16(), 429 | 503);
647 if !throttled || !idempotent || retries_left == 0 {
648 return Ok(response);
649 }
650 let stated = retry_after(response.headers(), &*self.clock);
651 let Some(delay) = next_delay(policy, waited, attempt, stated) else {
652 // The provider's pause (or the schedule) no longer
653 // fits the total: hand the caller the 429/503
654 // as-is, its `Retry-After` header intact.
655 return Ok(response);
656 };
657 (self.sleeper)(delay).await;
658 waited += delay;
659 retries_left -= 1;
660 attempt += 1;
661 }
662 Err(HttpError::Transport(detail)) if idempotent && retries_left > 0 => {
663 let Some(delay) = next_delay(policy, waited, attempt, None) else {
664 return Err(HttpError::Transport(detail));
665 };
666 (self.sleeper)(delay).await;
667 waited += delay;
668 retries_left -= 1;
669 attempt += 1;
670 }
671 // Deadline, size cap, a refused destination, our own
672 // budget refusal, or weather on a request that may not be
673 // retried: the caller hears it immediately.
674 Err(err) => return Err(err),
675 }
676 }
677 }
678}
679
680#[async_trait]
681impl HttpClient for BudgetedHttpClient {
682 async fn send(
683 &self,
684 request: http::Request<Bytes>,
685 ) -> Result<http::Response<Bytes>, HttpError> {
686 let Some(budget) = (self.router)(&request) else {
687 // Unrouted requests are not free — they are simply not ours to
688 // govern. The inner client's own bounds still apply.
689 return self.inner.send(request).await;
690 };
691 let idempotent = request_is_idempotent(&request);
692 self.acquire(&budget).await?;
693 self.spend_daily_quota(&budget).await?;
694 self.exchange_with(idempotent, || self.inner.send(request.clone()))
695 .await
696 }
697
698 async fn send_streaming(
699 &self,
700 request: http::Request<Bytes>,
701 ) -> Result<http::Response<ByteStream>, HttpError> {
702 let Some(budget) = (self.router)(&request) else {
703 // Unrouted requests are not free — they are simply not ours to
704 // govern. The inner client's own bounds still apply.
705 return self.inner.send_streaming(request).await;
706 };
707 let idempotent = request_is_idempotent(&request);
708 self.acquire(&budget).await?;
709 self.spend_daily_quota(&budget).await?;
710 // The budget is spent once, up front, exactly as `send` spends it;
711 // the retry loop covers the head only, so a stream never replays
712 // bytes that are already in flight.
713 self.exchange_with(idempotent, || self.inner.send_streaming(request.clone()))
714 .await
715 }
716}
717
718/// The timer the sleeps fall back to: `futures-timer`, which brings no
719/// executor and no runtime dependency. Its `Delay` reads the platform's
720/// monotonic clock, so on wasm32-unknown-unknown it still *compiles* (the
721/// wasm-safety gate) but cannot *fire* — std has no timer there — which is
722/// why a runtime on that target supplies its own sleeper.
723fn default_sleeper() -> Sleeper {
724 Arc::new(|after| Box::pin(async move { futures_timer::Delay::new(after).await }))
725}
726
727/// The safe-method default behind [`Idempotent`]'s absence. `Method`'s
728/// names are its own constants, not enum variants, so the comparison goes
729/// through `as_str` — the wire spelling is uppercase on every path here.
730fn default_method_idempotent(method: &http::Method) -> bool {
731 matches!(method.as_str(), "GET" | "HEAD" | "OPTIONS")
732}
733
734/// The request's idempotency: the [`Idempotent`] extension when present,
735/// the method's safety otherwise.
736fn request_is_idempotent(request: &http::Request<Bytes>) -> bool {
737 request.extensions().get::<Idempotent>().map_or_else(
738 || default_method_idempotent(request.method()),
739 |idempotent| idempotent.0,
740 )
741}
742
743/// The epoch-millisecond reading the bucket table stores, floored so the
744/// arithmetic stays integral (pre-1970 clocks included). Saturates at the
745/// `i64` edge, the way a counter does in `usage.rs`.
746fn epoch_millis(at: time::OffsetDateTime) -> i64 {
747 i64::try_from(at.unix_timestamp_nanos().div_euclid(1_000_000)).unwrap_or(i64::MAX)
748}
749
750/// The bucket level as a read-back row holds it. The write binds a double
751/// and rusqlite and Postgres answer with one, but D1's JSON bridge hands a
752/// safe integral JS number back as an integer — serde-wasm-bindgen visits
753/// whole numbers as `BigInt` — and `Row::get::<f64>` on an integer is
754/// `None`. Without the integral fallback, a bucket parked at exactly
755/// `9.0` or `0.0` reads as missing, and the advisory read treats a spent
756/// bucket as fresh. The integer shapes `Row` carries are the signed ones
757/// (`TinyInt` through `BigInt`), which one `i64` read covers; token
758/// counts stay far below the `f64` exact-integer range, so the widening
759/// cast loses nothing.
760#[expect(
761 clippy::cast_precision_loss,
762 reason = "token counts stay far below 2^53, the exact f64 integer range"
763)]
764fn tokens_of(row: &Row) -> Option<f64> {
765 row.get::<f64>("tokens")
766 .or_else(|| row.get::<i64>("tokens").map(|tokens| tokens as f64))
767}
768
769/// Milliseconds between `from_ms` and `to_ms`, as seconds, never negative —
770/// a clock that steps backwards must not *un*-refill a bucket.
771#[expect(
772 clippy::cast_precision_loss,
773 reason = "millisecond deltas stay far below 2^53"
774)]
775fn elapsed_seconds(from_ms: i64, to_ms: i64) -> f64 {
776 (to_ms - from_ms).max(0) as f64 / 1_000.0
777}
778
779/// The bucket level at `now_ms`: what was stored, plus the elapsed refill,
780/// clamped to capacity. A backwards clock leaves the level where it was.
781fn refilled_tokens(
782 stored: f64,
783 updated_ms: i64,
784 now_ms: i64,
785 refill_per_sec: f64,
786 capacity: f64,
787) -> f64 {
788 (stored + refill_per_sec * elapsed_seconds(updated_ms, now_ms)).min(capacity)
789}
790
791/// How long until the bucket can carry `cost`: zero when it already can,
792/// unrepresentable when it never will (`refill_per_sec` of zero), the
793/// missing tokens over the refill rate otherwise.
794fn pacing_retry_in(refilled: f64, cost: f64, refill_per_sec: f64) -> Duration {
795 if cost <= refilled {
796 return Duration::ZERO;
797 }
798 if refill_per_sec <= 0.0 {
799 return Duration::MAX;
800 }
801 Duration::try_from_secs_f64((cost - refilled) / refill_per_sec).unwrap_or(Duration::MAX)
802}
803
804/// The computed backoff for `attempt` (0-based): `base` doubled each time,
805/// clamped to `max_delay`. Saturating, so an absurd attempt count pins at
806/// the clamp instead of panicking.
807fn backoff_delay(base: Duration, max_delay: Duration, attempt: u32) -> Duration {
808 base.saturating_mul(2_u32.saturating_pow(attempt))
809 .min(max_delay)
810}
811
812/// The next wait: the upstream's stated `Retry-After` when it gave one
813/// (honoured up to what the total still allows), the exponential schedule
814/// otherwise, never past `max_total`. `None` — nothing left to wait with —
815/// means the caller keeps the response it has.
816fn next_delay(
817 policy: RetryPolicy,
818 waited: Duration,
819 attempt: u32,
820 stated: Option<Duration>,
821) -> Option<Duration> {
822 let remaining = policy.max_total().checked_sub(waited)?;
823 if remaining.is_zero() {
824 return None;
825 }
826 Some(match stated {
827 Some(stated) => stated.min(remaining),
828 None => backoff_delay(policy.base(), policy.max_delay(), attempt).min(remaining),
829 })
830}
831
832/// `ceil(limit * 80 / 100)` in integers, exact for every `u64` limit
833/// (saturating, for the limits no day will ever reach).
834#[must_use]
835fn warning_threshold(limit: u64) -> u64 {
836 limit
837 .saturating_mul(QUOTA_WARNING_NUMERATOR)
838 .div_ceil(QUOTA_WARNING_DENOMINATOR)
839}
840
841/// A budget-store failure as a transport error. `DbError`'s `Display` is
842/// already scrubbed (issue #135), so the key and the driver's words ride
843/// safely.
844fn store_failed(what: &str, err: &crate::DbError) -> HttpError {
845 HttpError::Transport(format!("budget store for {what} failed: {err}"))
846}
847
848/// The atomic refill-and-spend: one guarded upsert, one row changed on
849/// admission, none when the bucket cannot carry `cost` — the same
850/// rows-affected contract [`Usage::consume`] is built on, which is what
851/// SQLite, D1 and Postgres all report identically. The fresh-key arm seeds
852/// a full bucket minus this request's spend, guarded so a cost above
853/// capacity seeds nothing at all; the conflict arm refills from
854/// `updated_ms`, clamps to capacity with `CASE` (never `MIN`/`LEAST`),
855/// spends, and re-checks the guard against the refilled level, so two
856/// concurrent callers cannot both take the last token. The elapsed
857/// milliseconds pass through their own `CASE` — the SQL twin of
858/// [`elapsed_seconds`]: an isolate whose clock runs ahead writes a future
859/// `updated_ms`, and without the clamp every other isolate's negative
860/// elapsed would both shrink the guard (a refusal could pass) and burn
861/// `rate × skew` tokens off the level.
862fn acquire_sql(table: &str) -> String {
863 // `CASE WHEN`, never a `GREATEST`: the engines disagree on the
864 // spelling, and the scalar `max(a, b)` is SQLite-only.
865 let elapsed =
866 format!("CASE WHEN ? >= {table}.updated_ms THEN (? - {table}.updated_ms) ELSE 0 END");
867 format!(
868 "INSERT INTO {table} (bucket, tokens, updated_ms)\n\
869 SELECT ?, ? - ?, ? WHERE ? >= ?\n\
870 ON CONFLICT (bucket) DO UPDATE\n\
871 SET tokens = CASE\n\
872 WHEN {table}.tokens + ? * {elapsed} / 1000.0 > ?\n\
873 THEN ? - ?\n\
874 ELSE {table}.tokens + ? * {elapsed} / 1000.0 - ?\n\
875 END,\n\
876 updated_ms = ?\n\
877 WHERE {table}.tokens + ? * {elapsed} / 1000.0 >= ?"
878 )
879}
880
881/// The advisory read behind a failed spend: one key's level and stamp.
882fn bucket_read_sql(table: &str) -> String {
883 format!("SELECT tokens, updated_ms FROM {table} WHERE bucket = ?")
884}
885
886/// The quota-alert emission claim: one guarded insert, conflict-do-nothing,
887/// whose rows-affected answer is the whole at-most-once contract — `1` the
888/// caller owns the emission, `0` someone else already made it.
889fn claim_statement(
890 table: &str,
891 claimant: &str,
892 kind: &str,
893 period: &str,
894 claimed_ms: i64,
895) -> Statement {
896 Statement::with_values(
897 format!(
898 "INSERT INTO {table} (subject, period, kind, claimed_ms)\n\
899 VALUES (?, ?, ?, ?)\n\
900 ON CONFLICT (subject, period, kind) DO NOTHING"
901 ),
902 vec![
903 claimant.to_owned().into(),
904 period.to_owned().into(),
905 kind.to_owned().into(),
906 claimed_ms.into(),
907 ],
908 )
909}
910
911/// [`acquire_sql`]'s twenty-one binds, in the order the statement reads
912/// them: the seed row (key, capacity − cost, now, guarded by
913/// capacity ≥ cost), then the refill-and-spend three times over — the
914/// clamp, the spend, and the guard — each refill term reading the elapsed
915/// milliseconds through a two-bind `CASE` that clamps a backwards clock to
916/// zero, because no dialect here has a cheaper way to say "refilled,
917/// clamped, minus cost" without repeating it.
918fn acquire_statement(
919 table: &str,
920 key: &str,
921 capacity: f64,
922 cost: f64,
923 refill_per_sec: f64,
924 now_ms: i64,
925) -> Statement {
926 Statement::with_values(
927 acquire_sql(table),
928 vec![
929 key.to_owned().into(), // seed: bucket
930 capacity.into(), // seed: capacity - cost
931 cost.into(), //
932 now_ms.into(), // seed: updated_ms
933 capacity.into(), // seed guard: capacity >= cost
934 cost.into(), //
935 refill_per_sec.into(), // clamp: rate
936 now_ms.into(), // clamp: elapsed CASE, the `when`
937 now_ms.into(), // clamp: elapsed CASE, the `then`
938 capacity.into(), // clamp: refilled > capacity
939 capacity.into(), // spend: THEN capacity - cost
940 cost.into(), //
941 refill_per_sec.into(), // spend: ELSE rate
942 now_ms.into(), // spend: elapsed CASE, the `when`
943 now_ms.into(), // spend: elapsed CASE, the `then`
944 cost.into(), // spend: - cost
945 now_ms.into(), // stamp: updated_ms
946 refill_per_sec.into(), // guard: rate
947 now_ms.into(), // guard: elapsed CASE, the `when`
948 now_ms.into(), // guard: elapsed CASE, the `then`
949 cost.into(), // guard: >= cost
950 ],
951 )
952}
953
954#[cfg(test)]
955mod tests {
956 // Every float compared below is exactly representable (halves, wholes,
957 // tenths the arithmetic constructs by scaling by 1000), so `==` is the
958 // precise assertion, not a rounded one.
959 #![allow(clippy::float_cmp)]
960
961 use super::*;
962 use crate::lint_portable_sql;
963
964 // --- pure helpers ---------------------------------------------------
965
966 /// `elapsed_seconds` drives every refill; a backwards clock must read
967 /// as "no time passed", never as negative refill.
968 #[test]
969 fn elapsed_seconds_is_the_ms_delta_never_negative() {
970 assert_eq!(elapsed_seconds(1_000, 4_500), 3.5);
971 assert_eq!(elapsed_seconds(1_000, 1_000), 0.0);
972 assert_eq!(elapsed_seconds(5_000, 1_000), 0.0, "backwards clock");
973 }
974
975 /// The bucket level: stored plus refill, clamped at capacity, and a
976 /// backwards clock leaves the level where the row left it.
977 #[test]
978 fn refilled_tokens_clamps_at_capacity_and_ignores_backwards_clocks() {
979 assert_eq!(refilled_tokens(0.5, 0, 1_000, 1.0, 10.0), 1.5);
980 assert_eq!(
981 refilled_tokens(9.0, 0, 10_000, 1.0, 10.0),
982 10.0,
983 "the clamp, not 19"
984 );
985 assert_eq!(
986 refilled_tokens(4.0, 5_000, 1_000, 1.0, 10.0),
987 4.0,
988 "backwards clock: no un-refill"
989 );
990 }
991
992 /// `pacing_retry_in` is what a refusal reports as `retry_after`: zero
993 /// when the request fits, unrepresentable when nothing will ever
994 /// refill, the missing tokens over the rate otherwise.
995 #[test]
996 fn pacing_retry_in_scales_with_the_missing_tokens() {
997 assert_eq!(pacing_retry_in(5.0, 1.0, 1.0), Duration::ZERO);
998 assert_eq!(pacing_retry_in(0.0, 1.0, 0.0), Duration::MAX);
999 assert_eq!(pacing_retry_in(1.0, 2.0, 0.5), Duration::from_secs(2));
1000 assert_eq!(pacing_retry_in(9.0, 10.0, 4.0), Duration::from_millis(250));
1001 }
1002
1003 /// The backoff schedule: doubles from base, clamped at `max_delay`,
1004 /// saturating on absurd attempt counts.
1005 #[test]
1006 fn backoff_doubles_then_clamps() {
1007 let (base, max) = (Duration::from_millis(200), Duration::from_secs(5));
1008 assert_eq!(backoff_delay(base, max, 0), Duration::from_millis(200));
1009 assert_eq!(backoff_delay(base, max, 1), Duration::from_millis(400));
1010 assert_eq!(backoff_delay(base, max, 2), Duration::from_millis(800));
1011 assert_eq!(backoff_delay(base, max, 3), Duration::from_millis(1_600));
1012 assert_eq!(
1013 backoff_delay(base, max, 4),
1014 Duration::from_millis(3_200),
1015 "still under the clamp"
1016 );
1017 assert_eq!(
1018 backoff_delay(base, max, 5),
1019 Duration::from_secs(5),
1020 "clamped"
1021 );
1022 assert_eq!(backoff_delay(base, max, u32::MAX), Duration::from_secs(5));
1023 }
1024
1025 /// `next_delay` is the exchange's whole sense of time: stated
1026 /// `Retry-After` wins over the schedule, everything clamps to what the
1027 /// total still allows, and an exhausted total is `None` — no retry.
1028 #[test]
1029 fn next_delay_honours_the_stated_pause_within_the_total() {
1030 let policy = RetryPolicy::default();
1031 assert_eq!(
1032 next_delay(policy, Duration::ZERO, 0, None),
1033 Some(Duration::from_millis(200))
1034 );
1035 // Stated beats schedule; over max_delay but under max_total, so it
1036 // is honoured anyway.
1037 assert_eq!(
1038 next_delay(policy, Duration::ZERO, 0, Some(Duration::from_secs(9))),
1039 Some(Duration::from_secs(9))
1040 );
1041 // Stated clamps to the remaining total.
1042 assert_eq!(
1043 next_delay(
1044 policy,
1045 Duration::from_secs(10),
1046 0,
1047 Some(Duration::from_secs(9))
1048 ),
1049 Some(Duration::from_secs(5))
1050 );
1051 // The schedule never spends more than the remaining total: with
1052 // 1 s left, its 200 ms wait fits and is what comes back.
1053 assert_eq!(
1054 next_delay(policy, Duration::from_secs(14), 0, None),
1055 Some(Duration::from_millis(200))
1056 );
1057 // A schedule longer than what is left is cut to it.
1058 assert_eq!(
1059 next_delay(policy, Duration::from_secs(14), 4, None),
1060 Some(Duration::from_secs(1))
1061 );
1062 // Exactly at the total, and past it: nothing left to wait with.
1063 assert_eq!(next_delay(policy, Duration::from_secs(15), 0, None), None);
1064 assert_eq!(
1065 next_delay(policy, Duration::from_secs(16), 0, Some(Duration::ZERO)),
1066 None
1067 );
1068 // A total with headroom and a stated "now": retry immediately.
1069 assert_eq!(
1070 next_delay(policy, Duration::ZERO, 0, Some(Duration::ZERO)),
1071 Some(Duration::ZERO)
1072 );
1073 }
1074
1075 /// The 80% mark, exact in integers: `ceil(limit * 4 / 5)`, saturating.
1076 #[test]
1077 fn warning_threshold_is_the_exact_eighty_percent_mark() {
1078 assert_eq!(warning_threshold(10), 8);
1079 assert_eq!(warning_threshold(5), 4);
1080 assert_eq!(
1081 warning_threshold(3),
1082 3,
1083 "ceil(2.4) is 3 — the small-quota case"
1084 );
1085 assert_eq!(warning_threshold(1), 1);
1086 assert_eq!(warning_threshold(10_000), 8_000);
1087 // At the edge, the multiply saturates and the ceil-divide lands a
1088 // fifth of the range up — monotone and panic-free, which is all a
1089 // quota no day will reach owes anyone.
1090 assert_eq!(warning_threshold(u64::MAX), u64::MAX.div_ceil(5));
1091 }
1092
1093 /// Idempotency: safe methods by default, the [`Idempotent`] extension
1094 /// overrides either way.
1095 #[test]
1096 fn idempotency_defaults_to_safe_methods_and_the_extension_overrides() {
1097 let request = |method: http::Method| {
1098 http::Request::builder()
1099 .method(method)
1100 .body(Bytes::new())
1101 .expect("test request")
1102 };
1103 assert!(request_is_idempotent(&request(http::Method::GET)));
1104 assert!(request_is_idempotent(&request(http::Method::HEAD)));
1105 assert!(request_is_idempotent(&request(http::Method::OPTIONS)));
1106 assert!(!request_is_idempotent(&request(http::Method::POST)));
1107 assert!(!request_is_idempotent(&request(http::Method::DELETE)));
1108
1109 let mut vouched = request(http::Method::POST);
1110 vouched.extensions_mut().insert(Idempotent(true));
1111 assert!(request_is_idempotent(&vouched), "the caller vouches");
1112
1113 let mut withheld = request(http::Method::GET);
1114 withheld.extensions_mut().insert(Idempotent(false));
1115 assert!(!request_is_idempotent(&withheld), "the caller withholds");
1116 }
1117
1118 /// The DDL: names the table and passes the portable-SQL lint, like
1119 /// `Usage`'s quota DDL beside it and the alert-claim DDL beside that.
1120 #[test]
1121 fn the_bucket_ddl_is_portable_and_names_the_table() {
1122 let sql = BudgetedHttpClient::create_bucket_table_sql("http_bucket");
1123 assert!(sql.contains("CREATE TABLE IF NOT EXISTS http_bucket"));
1124 assert!(sql.contains("tokens DOUBLE PRECISION NOT NULL"));
1125 assert!(sql.contains("PRIMARY KEY (bucket)"));
1126 assert_eq!(lint_portable_sql(&sql), vec![], "DDL must be portable");
1127 let usage = Usage::new("http_usage").create_table_sql();
1128 assert_eq!(
1129 lint_portable_sql(&usage),
1130 vec![],
1131 "the quota DDL already is"
1132 );
1133 let alerts = BudgetedHttpClient::create_alert_table_sql("budget_alerts");
1134 assert!(alerts.contains("CREATE TABLE IF NOT EXISTS budget_alerts"));
1135 assert!(alerts.contains("PRIMARY KEY (subject, period, kind)"));
1136 assert_eq!(
1137 lint_portable_sql(&alerts),
1138 vec![],
1139 "the claim DDL must be portable too"
1140 );
1141 }
1142
1143 /// The alert-claim insert: four binds, conflict-do-nothing, the same
1144 /// rows-affected contract `SendCooldown`'s first-send arm is built on.
1145 #[test]
1146 fn the_alert_claim_inserts_one_guarded_row() {
1147 let statement = claim_statement(
1148 "budget_alerts",
1149 "http_usage:dexscreener",
1150 "exhausted",
1151 "2026-10-09T00:00:00Z",
1152 1_000,
1153 );
1154 assert!(
1155 statement
1156 .sql
1157 .contains("ON CONFLICT (subject, period, kind) DO NOTHING")
1158 );
1159 assert_eq!(statement.values.0.len(), 4);
1160 assert!(
1161 matches!(&statement.values.0[0], sea_query::Value::String(Some(who)) if who.as_str() == "http_usage:dexscreener"),
1162 "the claimant (quota table and key) is the first bind"
1163 );
1164 assert!(
1165 matches!(&statement.values.0[2], sea_query::Value::String(Some(kind)) if kind.as_str() == "exhausted"),
1166 "the kind is the third bind"
1167 );
1168 assert!(
1169 matches!(statement.values.0[3], sea_query::Value::BigInt(Some(v)) if v == 1_000),
1170 "the stamp binds as a big integer"
1171 );
1172 }
1173
1174 /// The acquire statement: twenty-one binds in the order the SQL reads
1175 /// them, doubles where the table says so and the stamp as a big
1176 /// integer, with `CASE` doing the clamp (never `MIN`/`LEAST`) — both
1177 /// the capacity clamp and, twice per refill term, the backwards-clock
1178 /// clamp on the elapsed milliseconds. The count is the SQL's, pinned
1179 /// here so a reordered clause fails this test before it fails a
1180 /// venture's request.
1181 #[test]
1182 fn the_acquire_statement_binds_the_seed_and_three_refill_passes() {
1183 let statement = acquire_statement("http_bucket", "dexscreener", 60.0, 1.0, 2.0, 1_000);
1184 assert!(statement.sql.contains("ON CONFLICT (bucket) DO UPDATE"));
1185 assert!(statement.sql.contains("CASE"));
1186 assert!(!statement.sql.contains("MIN("));
1187 assert!(!statement.sql.contains("LEAST("));
1188 // The backwards-clock clamp, once per refill term: the SQL twin of
1189 // `elapsed_seconds`, present in the clamp, the spend and the guard.
1190 assert_eq!(
1191 statement
1192 .sql
1193 .matches("CASE WHEN ? >= http_bucket.updated_ms")
1194 .count(),
1195 3,
1196 "each of the three refill terms clamps the elapsed time"
1197 );
1198 assert_eq!(statement.values.0.len(), 21);
1199 assert!(
1200 matches!(statement.values.0[1], sea_query::Value::Double(Some(v)) if v == 60.0),
1201 "capacity binds as a double"
1202 );
1203 assert!(
1204 matches!(statement.values.0[3], sea_query::Value::BigInt(Some(v)) if v == 1_000),
1205 "the stamp binds as a big integer"
1206 );
1207 assert!(
1208 matches!(&statement.values.0[0], sea_query::Value::String(Some(key)) if key.as_str() == "dexscreener"),
1209 "the key is the first bind"
1210 );
1211 // The first refill term's elapsed CASE reads the clock twice (the
1212 // `when` and the `then`), both as the big integer the column is.
1213 assert!(
1214 matches!(statement.values.0[7], sea_query::Value::BigInt(Some(v)) if v == 1_000),
1215 "the clamp's `when` binds the clock"
1216 );
1217 assert!(
1218 matches!(statement.values.0[8], sea_query::Value::BigInt(Some(v)) if v == 1_000),
1219 "the clamp's `then` binds the clock again"
1220 );
1221 }
1222
1223 /// The advisory read's level: the doubles the row carries natively,
1224 /// and the integral fallback D1 needs — its JSON bridge hands a safe
1225 /// integral JS number back as an integer, on which `Row::get::<f64>`
1226 /// is `None`, reading a bucket parked at exactly `9.0` as missing.
1227 #[test]
1228 fn the_bucket_level_reads_as_doubles_and_as_the_integral_shapes_d1_hands_back() {
1229 let row = |value: sea_query::Value| Row::new(vec![("tokens".to_owned(), value)]);
1230 assert_eq!(tokens_of(&row(9.5.into())), Some(9.5), "double, as bound");
1231 assert_eq!(
1232 tokens_of(&row(sea_query::Value::Float(Some(9.5)))),
1233 Some(9.5),
1234 "the single-precision shape some drivers answer with"
1235 );
1236 assert_eq!(
1237 tokens_of(&row(9_i64.into())),
1238 Some(9.0),
1239 "D1's integral numbers: the fallback, not `None`"
1240 );
1241 assert_eq!(
1242 tokens_of(&row(0_i64.into())),
1243 Some(0.0),
1244 "the spent bucket, exactly zero, reads as zero"
1245 );
1246 assert_eq!(
1247 tokens_of(&row(sea_query::Value::Double(None))),
1248 None,
1249 "NULL is still missing"
1250 );
1251 assert_eq!(
1252 tokens_of(&row("9".to_owned().into())),
1253 None,
1254 "a non-numeric column is unrepresentable, as ever"
1255 );
1256 }
1257
1258 /// The advisory read: one key's row, nothing else.
1259 #[test]
1260 fn the_bucket_read_targets_one_key() {
1261 assert_eq!(
1262 bucket_read_sql("http_bucket"),
1263 "SELECT tokens, updated_ms FROM http_bucket WHERE bucket = ?"
1264 );
1265 }
1266
1267 /// The policy's defaults are the ones the issue names, and the
1268 /// builders change exactly what they name.
1269 #[test]
1270 fn the_retry_policy_defaults_and_builds() {
1271 let policy = RetryPolicy::default();
1272 assert_eq!(policy.max_retries(), 2);
1273 assert_eq!(policy.base(), Duration::from_millis(200));
1274 assert_eq!(policy.max_delay(), Duration::from_secs(5));
1275 assert_eq!(policy.max_total(), Duration::from_secs(15));
1276 let tuned = policy
1277 .with_max_retries(5)
1278 .with_base(Duration::from_millis(50))
1279 .with_max_delay(Duration::from_secs(2))
1280 .with_max_total(Duration::from_secs(30));
1281 assert_eq!(tuned.max_retries(), 5);
1282 assert_eq!(tuned.base(), Duration::from_millis(50));
1283 assert_eq!(tuned.max_delay(), Duration::from_secs(2));
1284 assert_eq!(tuned.max_total(), Duration::from_secs(30));
1285 // `Copy` semantics: the default was not disturbed.
1286 assert_eq!(policy.max_retries(), 2);
1287 }
1288
1289 /// The epoch stamp: whole milliseconds since the epoch, floored, far
1290 /// from the `i64` edge the saturating cast guards.
1291 #[test]
1292 fn epoch_millis_floors_to_whole_milliseconds() {
1293 let at = |nanos: i64| time::OffsetDateTime::UNIX_EPOCH + time::Duration::nanoseconds(nanos);
1294 assert_eq!(epoch_millis(at(1_500_000)), 1, "1.5 ms floors, not rounds");
1295 assert_eq!(epoch_millis(at(1_999_999)), 1);
1296 assert_eq!(epoch_millis(at(2_000_000)), 2);
1297 assert_eq!(epoch_millis(at(0)), 0);
1298 assert_eq!(
1299 epoch_millis(at(-1_500_000)),
1300 -2,
1301 "floors below the epoch too"
1302 );
1303 }
1304}