Crate throttle_net 

throttle-net

Outbound throttling and resilience. Where rate-net protects your service from being overwhelmed (inbound), throttle-net protects your service from overwhelming the downstreams it calls — and from being banned by them. The defining operation is therefore to wait, not to reject: you pace your own outbound work rather than dropping someone else’s request.

throttle-net does not reimplement token-bucket accounting. It consumes better-bucket for that and reads time from clock-lib, then builds the waiting, cost-aware, composable surface on top. It is the outbound companion to rate-net.

Status

Pre-1.0 (v0.4). The limiter and resilience surface so far: the Limiter trait, the Throttle token bucket and the exact SlidingWindowLog, each with a waiting cost-aware acquire; the composites — Hybrid (must pass all), MultiLimiter (multi-dimensional budgets), PerKey (independent per-key state, bounded memory), and Layered (global / per-key / per-endpoint scopes); standalone Retry/Backoff with jittered backoff and Retry-After parsing; and the resilience layer — a CircuitBreaker that wraps any limiter and fails fast, and a deadline-aware, priority Queue. Adaptive limiting and provider presets land across the rest of the 0.x series. The public API is frozen at 1.0.

use throttle_net::Throttle;

// 100 requests per second, bursting up to 100.
let throttle = Throttle::per_second(100);

// Pace an outbound call: returns as soon as a token is free.
throttle.acquire().await?;
// ... call the downstream ...

When you would rather not wait, ask without blocking:

use throttle_net::Throttle;

let throttle = Throttle::per_second(100);
if throttle.try_acquire() {
    // a token was free — send now
}

Design goals

• Wait by default. The Tier-1 acquire paces the caller; try_acquire is there when you need the non-blocking answer.
• Cost-aware. Not every request weighs one unit. acquire_with_cost(n) takes n tokens at once — the basis for the multi-dimensional LLM budgets that arrive with the rest of v0.2.
• Lock-free accounting. Each acquire is a single atomic compare-and-swap in better-bucket; no lock sits on the path.
• Runtime-free core, lazy refill. Tokens accrue from a monotonic clock on access; there is no background timer thread, and the synchronous core has no async-runtime dependency.
• Composable. Every limiter is one Limiter; composites combine them without the call site changing.

Feature flags

Feature	Default	Description
std	yes	Standard library. Gates the limiter surface. With it off the crate is no_std and exposes only VERSION.
tokio	yes	The waiting acquire surface, driven by tokio’s timer. Implies std.

See docs/API.md for the full feature matrix as later phases land.

Structs

Backoffstd

A backoff policy: a base curve plus a jitter mode and a delay ceiling.

BackoffIterstd

A live delay sequence produced by a Backoff.

CircuitBreakercircuit-breaker

A circuit breaker wrapping a limiter L, timed by clock C.

CircuitBreakerBuildercircuit-breaker

Builder for a CircuitBreaker.

Evictionstd

How a PerKey limiter bounds the memory its per-key state can occupy.

Hybridstd

Several limiters combined so a request must satisfy all of them.

HybridBuilderstd

Builder for a Hybrid limiter.

Layeredstd

Several scopes of limiting stacked so a request must clear every one.

LayeredBuilderstd

Builder for a Layered limiter.

ManualClockstd

A clock under your control, for deterministic testing.

MultiLimiterstd

A limiter with several named dimensions, each metered independently.

MultiLimiterBuilderstd

Builder for a MultiLimiter.

PerKeystd

A throttle that keeps independent state per key.

Permitcircuit-breaker

A reserved permission to make one protected call.

Queuetokio

A bounded, deadline-aware, priority queue fronting a limiter L, keyed by K for fairness and timed by clock C.

QueueBuildertokio

Builder for a Queue.

Retrystd

A retry policy: a Backoff, an attempt ceiling, and whether to honor a server’s Retry-After.

SlidingWindowLogstd

An exact sliding-window-log rate limiter: at most limit units in any trailing window of window.

SystemClockstd

A clock backed by the operating system.

Throttlestd

A single outbound throttle backed by a token bucket.

Enums

BreakerStatecircuit-breaker

The breaker’s current state, as a snapshot.

Decisionstd

What happened when a limiter was asked for tokens without waiting.

Jitterstd

How retry delays are randomized to avoid synchronized retries.

Overflowtokio

What a full queue does with a new request.

RetryActionstd

What to do with an error a retried operation returned.

ThrottleErrorstd

An acquisition that cannot complete.

Tripcircuit-breaker

The condition under which a closed breaker trips open.

Constants

DEFAULT_MAX_KEYSstd

The default key-capacity cap, applied unless overridden.

VERSION

The version of this crate, from Cargo.toml.

Traits

Clockstd

A source of time.

Limiterstd

The contract a limiter satisfies: take a cost and report the Decision.

Functions

parse_retry_afterstd

Parses a Retry-After header value into a delay from now.

parse_retry_after_atstd

Parses a Retry-After value relative to an explicit current time, given as Unix seconds. The date form needs a reference point; this lets tests and clock-injecting callers supply one.

retry_if_retryablestd

Classifies an error_forge::ForgeError by its own retryability: retry when is_retryable is true, otherwise give up. A convenient classify argument for Retry::run.