throttle_net/lib.rs
1//! # throttle-net
2//!
3//! Outbound throttling and resilience. Where `rate-net` protects your service
4//! from being overwhelmed (inbound), throttle-net protects your service from
5//! *overwhelming* the downstreams it calls — and from being banned by them. The
6//! defining operation is therefore to **wait**, not to reject: you pace your own
7//! outbound work rather than dropping someone else's request.
8//!
9//! throttle-net does not reimplement token-bucket accounting. It consumes
10//! [`better-bucket`](https://crates.io/crates/better-bucket) for that and reads
11//! time from [`clock-lib`](https://crates.io/crates/clock-lib), then builds the
12//! waiting, cost-aware, composable surface on top. It is the outbound companion
13//! to [`rate-net`](https://crates.io/crates/rate-net).
14//!
15//! ## Status
16//!
17//! **Pre-1.0 (v0.4).** The limiter and resilience surface so far: the [`Limiter`]
18//! trait, the [`Throttle`] token bucket and the exact [`SlidingWindowLog`], each
19//! with a waiting cost-aware [`acquire`](Throttle::acquire); the composites —
20//! [`Hybrid`] (must pass all), [`MultiLimiter`] (multi-dimensional budgets),
21//! [`PerKey`] (independent per-key state, bounded memory), and [`Layered`]
22//! (global / per-key / per-endpoint scopes); standalone [`Retry`]/[`Backoff`]
23//! with jittered backoff and `Retry-After` parsing; and the resilience layer —
24//! a [`CircuitBreaker`] that wraps any limiter and fails fast, and a deadline-aware,
25//! priority [`Queue`]. Adaptive limiting and provider presets land across the
26//! rest of the 0.x series. The public API is frozen at 1.0.
27//!
28//! ```
29//! # #[cfg(feature = "tokio")]
30//! # async fn run() -> Result<(), throttle_net::ThrottleError> {
31//! use throttle_net::Throttle;
32//!
33//! // 100 requests per second, bursting up to 100.
34//! let throttle = Throttle::per_second(100);
35//!
36//! // Pace an outbound call: returns as soon as a token is free.
37//! throttle.acquire().await?;
38//! // ... call the downstream ...
39//! # Ok(())
40//! # }
41//! ```
42//!
43//! When you would rather not wait, ask without blocking:
44//!
45//! ```
46//! # #[cfg(feature = "std")] {
47//! use throttle_net::Throttle;
48//!
49//! let throttle = Throttle::per_second(100);
50//! if throttle.try_acquire() {
51//! // a token was free — send now
52//! }
53//! # }
54//! ```
55//!
56//! ## Design goals
57//!
58//! - **Wait by default.** The Tier-1 [`acquire`](Throttle::acquire) paces the
59//! caller; [`try_acquire`](Throttle::try_acquire) is there when you need the
60//! non-blocking answer.
61//! - **Cost-aware.** Not every request weighs one unit. `acquire_with_cost(n)`
62//! takes `n` tokens at once — the basis for the multi-dimensional LLM budgets
63//! that arrive with the rest of v0.2.
64//! - **Lock-free accounting.** Each acquire is a single atomic compare-and-swap
65//! in `better-bucket`; no lock sits on the path.
66//! - **Runtime-free core, lazy refill.** Tokens accrue from a monotonic clock on
67//! access; there is no background timer thread, and the synchronous core has no
68//! async-runtime dependency.
69//! - **Composable.** Every limiter is one [`Limiter`]; composites combine them
70//! without the call site changing.
71//!
72//! ## Feature flags
73//!
74//! | Feature | Default | Description |
75//! |---------|---------|-------------|
76//! | `std` | yes | Standard library. Gates the limiter surface. With it off the crate is `no_std` and exposes only [`VERSION`]. |
77//! | `tokio` | yes | The waiting [`acquire`](Throttle::acquire) surface, driven by tokio's timer. Implies `std`. |
78//!
79//! See `docs/API.md` for the full feature matrix as later phases land.
80
81// `no_std` for the library build when `std` is off, but always link `std` under
82// `test` so the unit-test harness and dev-dependencies have what they need.
83#![cfg_attr(all(not(feature = "std"), not(test)), no_std)]
84#![cfg_attr(docsrs, feature(doc_cfg))]
85#![deny(missing_docs)]
86#![forbid(unsafe_code)]
87#![deny(unused_must_use)]
88#![deny(unused_results)]
89#![deny(clippy::unwrap_used)]
90#![deny(clippy::expect_used)]
91#![deny(clippy::todo)]
92#![deny(clippy::unimplemented)]
93#![deny(clippy::print_stdout)]
94#![deny(clippy::print_stderr)]
95#![deny(clippy::dbg_macro)]
96#![deny(clippy::unreachable)]
97#![deny(clippy::undocumented_unsafe_blocks)]
98
99// The limiter surface requires the standard library (the clock-driven token
100// bucket and the domain error type). With `std` off the crate is `no_std` and
101// exposes only `VERSION`.
102#[cfg(feature = "std")]
103mod backoff;
104#[cfg(feature = "circuit-breaker")]
105mod circuit;
106#[cfg(feature = "std")]
107mod decision;
108#[cfg(feature = "std")]
109mod error;
110#[cfg(feature = "std")]
111mod eviction;
112#[cfg(feature = "std")]
113mod hybrid;
114#[cfg(feature = "std")]
115mod layered;
116#[cfg(feature = "std")]
117mod limiter;
118#[cfg(feature = "std")]
119mod multi;
120#[cfg(feature = "std")]
121mod perkey;
122#[cfg(feature = "tokio")]
123mod queue;
124#[cfg(feature = "std")]
125mod retry;
126#[cfg(feature = "std")]
127mod retry_after;
128#[cfg(feature = "std")]
129mod sliding;
130#[cfg(feature = "std")]
131mod throttle;
132
133#[cfg(feature = "std")]
134pub use crate::backoff::{Backoff, BackoffIter, Jitter};
135#[cfg(feature = "circuit-breaker")]
136pub use crate::circuit::{BreakerState, CircuitBreaker, CircuitBreakerBuilder, Permit, Trip};
137#[cfg(feature = "std")]
138pub use crate::decision::Decision;
139#[cfg(feature = "std")]
140pub use crate::error::ThrottleError;
141#[cfg(feature = "std")]
142pub use crate::eviction::{DEFAULT_MAX_KEYS, Eviction};
143#[cfg(feature = "std")]
144pub use crate::hybrid::{Hybrid, HybridBuilder};
145#[cfg(feature = "std")]
146pub use crate::layered::{Layered, LayeredBuilder};
147#[cfg(feature = "std")]
148pub use crate::limiter::Limiter;
149#[cfg(feature = "std")]
150pub use crate::multi::{MultiLimiter, MultiLimiterBuilder};
151#[cfg(feature = "std")]
152pub use crate::perkey::PerKey;
153#[cfg(feature = "tokio")]
154pub use crate::queue::{Overflow, Queue, QueueBuilder};
155#[cfg(feature = "std")]
156pub use crate::retry::{Retry, RetryAction, retry_if_retryable};
157#[cfg(feature = "std")]
158pub use crate::retry_after::{parse_retry_after, parse_retry_after_at};
159#[cfg(feature = "std")]
160pub use crate::sliding::SlidingWindowLog;
161#[cfg(feature = "std")]
162pub use crate::throttle::Throttle;
163
164// The clock seam is part of the public API: [`Throttle::with_clock`] and the
165// per-key/composite `with_clock` methods take any [`Clock`], and tests drive a
166// [`ManualClock`]. Re-exported so callers need not depend on `clock-lib` directly.
167#[cfg(feature = "std")]
168pub use clock_lib::{Clock, ManualClock, SystemClock};
169
170/// The version of this crate, from `Cargo.toml`.
171///
172/// # Examples
173///
174/// ```
175/// assert!(!throttle_net::VERSION.is_empty());
176/// ```
177pub const VERSION: &str = env!("CARGO_PKG_VERSION");