throttle_net/lib.rs
1//! # throttle-net
2//!
3//! Outbound throttling and resilience. Where `rate-net` protects your service
4//! from being overwhelmed (inbound), throttle-net protects your service from
5//! *overwhelming* the downstreams it calls — and from being banned by them. The
6//! defining operation is therefore to **wait**, not to reject: you pace your own
7//! outbound work rather than dropping someone else's request.
8//!
9//! throttle-net does not reimplement token-bucket accounting. It consumes
10//! [`better-bucket`](https://crates.io/crates/better-bucket) for that and reads
11//! time from [`clock-lib`](https://crates.io/crates/clock-lib), then builds the
12//! waiting, cost-aware, composable surface on top. It is the outbound companion
13//! to [`rate-net`](https://crates.io/crates/rate-net).
14//!
15//! ## Status
16//!
17//! **Pre-1.0 (v0.3).** The limiter and resilience surface so far: the
18//! [`Limiter`] trait, the [`Throttle`] token bucket with its waiting cost-aware
19//! [`acquire`](Throttle::acquire), the composites — [`Hybrid`] (must pass all),
20//! [`MultiLimiter`] (multi-dimensional budgets), [`PerKey`] (independent per-key
21//! state, bounded memory), and [`Layered`] (global / per-key / per-endpoint
22//! scopes) — and standalone [`Retry`]/[`Backoff`] with jittered backoff and
23//! `Retry-After` parsing. Circuit breakers and adaptive limiting land across the
24//! rest of the 0.x series. The public API is frozen at 1.0.
25//!
26//! ```
27//! # #[cfg(feature = "tokio")]
28//! # async fn run() -> Result<(), throttle_net::ThrottleError> {
29//! use throttle_net::Throttle;
30//!
31//! // 100 requests per second, bursting up to 100.
32//! let throttle = Throttle::per_second(100);
33//!
34//! // Pace an outbound call: returns as soon as a token is free.
35//! throttle.acquire().await?;
36//! // ... call the downstream ...
37//! # Ok(())
38//! # }
39//! ```
40//!
41//! When you would rather not wait, ask without blocking:
42//!
43//! ```
44//! # #[cfg(feature = "std")] {
45//! use throttle_net::Throttle;
46//!
47//! let throttle = Throttle::per_second(100);
48//! if throttle.try_acquire() {
49//! // a token was free — send now
50//! }
51//! # }
52//! ```
53//!
54//! ## Design goals
55//!
56//! - **Wait by default.** The Tier-1 [`acquire`](Throttle::acquire) paces the
57//! caller; [`try_acquire`](Throttle::try_acquire) is there when you need the
58//! non-blocking answer.
59//! - **Cost-aware.** Not every request weighs one unit. `acquire_with_cost(n)`
60//! takes `n` tokens at once — the basis for the multi-dimensional LLM budgets
61//! that arrive with the rest of v0.2.
62//! - **Lock-free accounting.** Each acquire is a single atomic compare-and-swap
63//! in `better-bucket`; no lock sits on the path.
64//! - **Runtime-free core, lazy refill.** Tokens accrue from a monotonic clock on
65//! access; there is no background timer thread, and the synchronous core has no
66//! async-runtime dependency.
67//! - **Composable.** Every limiter is one [`Limiter`]; composites combine them
68//! without the call site changing.
69//!
70//! ## Feature flags
71//!
72//! | Feature | Default | Description |
73//! |---------|---------|-------------|
74//! | `std` | yes | Standard library. Gates the limiter surface. With it off the crate is `no_std` and exposes only [`VERSION`]. |
75//! | `tokio` | yes | The waiting [`acquire`](Throttle::acquire) surface, driven by tokio's timer. Implies `std`. |
76//!
77//! See `docs/API.md` for the full feature matrix as later phases land.
78
79// `no_std` for the library build when `std` is off, but always link `std` under
80// `test` so the unit-test harness and dev-dependencies have what they need.
81#![cfg_attr(all(not(feature = "std"), not(test)), no_std)]
82#![cfg_attr(docsrs, feature(doc_cfg))]
83#![deny(missing_docs)]
84#![forbid(unsafe_code)]
85#![deny(unused_must_use)]
86#![deny(unused_results)]
87#![deny(clippy::unwrap_used)]
88#![deny(clippy::expect_used)]
89#![deny(clippy::todo)]
90#![deny(clippy::unimplemented)]
91#![deny(clippy::print_stdout)]
92#![deny(clippy::print_stderr)]
93#![deny(clippy::dbg_macro)]
94#![deny(clippy::unreachable)]
95#![deny(clippy::undocumented_unsafe_blocks)]
96
97// The limiter surface requires the standard library (the clock-driven token
98// bucket and the domain error type). With `std` off the crate is `no_std` and
99// exposes only `VERSION`.
100#[cfg(feature = "std")]
101mod backoff;
102#[cfg(feature = "std")]
103mod decision;
104#[cfg(feature = "std")]
105mod error;
106#[cfg(feature = "std")]
107mod eviction;
108#[cfg(feature = "std")]
109mod hybrid;
110#[cfg(feature = "std")]
111mod layered;
112#[cfg(feature = "std")]
113mod limiter;
114#[cfg(feature = "std")]
115mod multi;
116#[cfg(feature = "std")]
117mod perkey;
118#[cfg(feature = "std")]
119mod retry;
120#[cfg(feature = "std")]
121mod retry_after;
122#[cfg(feature = "std")]
123mod throttle;
124
125#[cfg(feature = "std")]
126pub use crate::backoff::{Backoff, BackoffIter, Jitter};
127#[cfg(feature = "std")]
128pub use crate::decision::Decision;
129#[cfg(feature = "std")]
130pub use crate::error::ThrottleError;
131#[cfg(feature = "std")]
132pub use crate::eviction::{DEFAULT_MAX_KEYS, Eviction};
133#[cfg(feature = "std")]
134pub use crate::hybrid::{Hybrid, HybridBuilder};
135#[cfg(feature = "std")]
136pub use crate::layered::{Layered, LayeredBuilder};
137#[cfg(feature = "std")]
138pub use crate::limiter::Limiter;
139#[cfg(feature = "std")]
140pub use crate::multi::{MultiLimiter, MultiLimiterBuilder};
141#[cfg(feature = "std")]
142pub use crate::perkey::PerKey;
143#[cfg(feature = "std")]
144pub use crate::retry::{Retry, RetryAction, retry_if_retryable};
145#[cfg(feature = "std")]
146pub use crate::retry_after::{parse_retry_after, parse_retry_after_at};
147#[cfg(feature = "std")]
148pub use crate::throttle::Throttle;
149
150// The clock seam is part of the public API: [`Throttle::with_clock`] and the
151// per-key/composite `with_clock` methods take any [`Clock`], and tests drive a
152// [`ManualClock`]. Re-exported so callers need not depend on `clock-lib` directly.
153#[cfg(feature = "std")]
154pub use clock_lib::{Clock, ManualClock, SystemClock};
155
156/// The version of this crate, from `Cargo.toml`.
157///
158/// # Examples
159///
160/// ```
161/// assert!(!throttle_net::VERSION.is_empty());
162/// ```
163pub const VERSION: &str = env!("CARGO_PKG_VERSION");