throttle_net/lib.rs
1//! # throttle-net
2//!
3//! Outbound throttling and resilience. Where `rate-net` protects your service
4//! from being overwhelmed (inbound), throttle-net protects your service from
5//! *overwhelming* the downstreams it calls — and from being banned by them. The
6//! defining operation is therefore to **wait**, not to reject: you pace your own
7//! outbound work rather than dropping someone else's request.
8//!
9//! throttle-net does not reimplement token-bucket accounting. It consumes
10//! [`better-bucket`](https://crates.io/crates/better-bucket) for that and reads
11//! time from [`clock-lib`](https://crates.io/crates/clock-lib), then builds the
12//! waiting, cost-aware, composable surface on top. It is the outbound companion
13//! to [`rate-net`](https://crates.io/crates/rate-net).
14//!
15//! ## Status
16//!
17//! **Pre-1.0 (v0.7).** The limiter and resilience surface so far: the [`Limiter`]
18//! trait, the [`Throttle`] token bucket and the exact [`SlidingWindowLog`], each
19//! with a waiting cost-aware [`acquire`](Throttle::acquire); the composites —
20//! [`Hybrid`] (must pass all), [`MultiLimiter`] (multi-dimensional budgets),
21//! [`PerKey`] (independent per-key state, bounded memory), and [`Layered`]
22//! (global / per-key / per-endpoint scopes); standalone [`Retry`]/[`Backoff`]
23//! with jittered backoff and `Retry-After` parsing; the resilience layer —
24//! a `CircuitBreaker` that wraps any limiter and fails fast (`circuit-breaker`
25//! feature), and a deadline-aware, priority [`Queue`]; adaptive concurrency —
26//! an `AdaptiveLimiter` that discovers the right in-flight limit from outcome
27//! feedback (`adaptive` feature); provider integration — response-header parsers
28//! with limiter sync (`provider`, `provider-headers` feature) and LLM tier
29//! `presets` (`provider-llm` feature); and observability — metrics and tracing
30//! events, feature-gated and zero-cost when off (`metrics`, `tracing` features).
31//! Runtime flexibility lands across the rest of the 0.x series. The public API is
32//! frozen at 1.0.
33//!
34//! ```
35//! # #[cfg(feature = "tokio")]
36//! # async fn run() -> Result<(), throttle_net::ThrottleError> {
37//! use throttle_net::Throttle;
38//!
39//! // 100 requests per second, bursting up to 100.
40//! let throttle = Throttle::per_second(100);
41//!
42//! // Pace an outbound call: returns as soon as a token is free.
43//! throttle.acquire().await?;
44//! // ... call the downstream ...
45//! # Ok(())
46//! # }
47//! ```
48//!
49//! When you would rather not wait, ask without blocking:
50//!
51//! ```
52//! # #[cfg(feature = "std")] {
53//! use throttle_net::Throttle;
54//!
55//! let throttle = Throttle::per_second(100);
56//! if throttle.try_acquire() {
57//! // a token was free — send now
58//! }
59//! # }
60//! ```
61//!
62//! ## Design goals
63//!
64//! - **Wait by default.** The Tier-1 [`acquire`](Throttle::acquire) paces the
65//! caller; [`try_acquire`](Throttle::try_acquire) is there when you need the
66//! non-blocking answer.
67//! - **Cost-aware.** Not every request weighs one unit. `acquire_with_cost(n)`
68//! takes `n` tokens at once — the basis for the multi-dimensional LLM budgets
69//! that arrive with the rest of v0.2.
70//! - **Lock-free accounting.** Each acquire is a single atomic compare-and-swap
71//! in `better-bucket`; no lock sits on the path.
72//! - **Runtime-free core, lazy refill.** Tokens accrue from a monotonic clock on
73//! access; there is no background timer thread, and the synchronous core has no
74//! async-runtime dependency.
75//! - **Composable.** Every limiter is one [`Limiter`]; composites combine them
76//! without the call site changing.
77//!
78//! ## Feature flags
79//!
80//! | Feature | Default | Description |
81//! |---------|---------|-------------|
82//! | `std` | yes | Standard library. Gates the limiter surface. With it off the crate is `no_std` and exposes only [`VERSION`]. |
83//! | `tokio` | yes | The waiting [`acquire`](Throttle::acquire) surface, driven by tokio's timer. Implies `std`. |
84//!
85//! See `docs/API.md` for the full feature matrix as later phases land.
86
87// `no_std` for the library build when `std` is off, but always link `std` under
88// `test` so the unit-test harness and dev-dependencies have what they need.
89#![cfg_attr(all(not(feature = "std"), not(test)), no_std)]
90#![cfg_attr(docsrs, feature(doc_cfg))]
91#![deny(missing_docs)]
92#![forbid(unsafe_code)]
93#![deny(unused_must_use)]
94#![deny(unused_results)]
95#![deny(clippy::unwrap_used)]
96#![deny(clippy::expect_used)]
97#![deny(clippy::todo)]
98#![deny(clippy::unimplemented)]
99#![deny(clippy::print_stdout)]
100#![deny(clippy::print_stderr)]
101#![deny(clippy::dbg_macro)]
102#![deny(clippy::unreachable)]
103#![deny(clippy::undocumented_unsafe_blocks)]
104
105// The limiter surface requires the standard library (the clock-driven token
106// bucket and the domain error type). With `std` off the crate is `no_std` and
107// exposes only `VERSION`.
108#[cfg(feature = "adaptive")]
109mod adaptive;
110#[cfg(feature = "std")]
111mod backoff;
112#[cfg(feature = "circuit-breaker")]
113mod circuit;
114#[cfg(feature = "std")]
115mod decision;
116#[cfg(feature = "std")]
117mod error;
118#[cfg(feature = "std")]
119mod eviction;
120#[cfg(feature = "std")]
121mod hybrid;
122#[cfg(feature = "std")]
123mod layered;
124#[cfg(feature = "std")]
125mod limiter;
126#[cfg(feature = "std")]
127mod multi;
128#[cfg(any(feature = "tokio", feature = "circuit-breaker", feature = "adaptive"))]
129mod obs;
130#[cfg(feature = "std")]
131mod perkey;
132#[cfg(feature = "provider-llm")]
133pub mod presets;
134#[cfg(feature = "provider-headers")]
135pub mod provider;
136#[cfg(feature = "tokio")]
137mod queue;
138#[cfg(feature = "std")]
139mod retry;
140#[cfg(feature = "std")]
141mod retry_after;
142#[cfg(feature = "std")]
143mod sliding;
144#[cfg(feature = "std")]
145mod throttle;
146#[cfg(feature = "std")]
147mod timeutil;
148
149#[cfg(feature = "adaptive")]
150pub use crate::adaptive::{
151 AdaptiveLimiter, AdaptiveLimiterBuilder, AdaptivePermit, AdaptiveStrategy, Aimd, Outcome, Vegas,
152};
153#[cfg(feature = "std")]
154pub use crate::backoff::{Backoff, BackoffIter, Jitter};
155#[cfg(feature = "circuit-breaker")]
156pub use crate::circuit::{BreakerState, CircuitBreaker, CircuitBreakerBuilder, Permit, Trip};
157#[cfg(feature = "std")]
158pub use crate::decision::Decision;
159#[cfg(feature = "std")]
160pub use crate::error::ThrottleError;
161#[cfg(feature = "std")]
162pub use crate::eviction::{DEFAULT_MAX_KEYS, Eviction};
163#[cfg(feature = "std")]
164pub use crate::hybrid::{Hybrid, HybridBuilder};
165#[cfg(feature = "std")]
166pub use crate::layered::{Layered, LayeredBuilder};
167#[cfg(feature = "std")]
168pub use crate::limiter::Limiter;
169#[cfg(feature = "std")]
170pub use crate::multi::{MultiLimiter, MultiLimiterBuilder};
171#[cfg(feature = "std")]
172pub use crate::perkey::PerKey;
173#[cfg(feature = "tokio")]
174pub use crate::queue::{Overflow, Queue, QueueBuilder};
175#[cfg(feature = "std")]
176pub use crate::retry::{Retry, RetryAction, retry_if_retryable};
177#[cfg(feature = "std")]
178pub use crate::retry_after::{parse_retry_after, parse_retry_after_at};
179#[cfg(feature = "std")]
180pub use crate::sliding::SlidingWindowLog;
181#[cfg(feature = "std")]
182pub use crate::throttle::Throttle;
183
184// The clock seam is part of the public API: [`Throttle::with_clock`] and the
185// per-key/composite `with_clock` methods take any [`Clock`], and tests drive a
186// [`ManualClock`]. Re-exported so callers need not depend on `clock-lib` directly.
187#[cfg(feature = "std")]
188pub use clock_lib::{Clock, ManualClock, SystemClock};
189
190/// The version of this crate, from `Cargo.toml`.
191///
192/// # Examples
193///
194/// ```
195/// assert!(!throttle_net::VERSION.is_empty());
196/// ```
197pub const VERSION: &str = env!("CARGO_PKG_VERSION");