Crate tokio_rate_limit 

tokio-rate-limit

High-performance rate limiting library with lock-free token accounting and sharded state management.

Features

• Lock-free token accounting: Atomic CAS on per-key buckets
• 256-shard state management: Lock-free concurrent hashmap (flurry)
• Pluggable algorithms: Token bucket, leaky bucket, probabilistic sampling
• Axum / Tonic middleware: Drop-in HTTP and gRPC rate limiting

Absolute throughput is hardware-specific; compare against governor with benches/comparison.rs on the machine you care about. The hot path still allocates on first sight of a key (the map insert).

Architecture

This library uses a hybrid approach for maximum performance:

• Token updates: True lock-free atomic compare-and-swap operations
• Key lookup: 256-shard lock-free concurrent hashmap (flurry) for near-linear multi-threaded scaling

This design provides excellent concurrency while maintaining per-key isolation.

Quick Start

use tokio_rate_limit::{RateLimiter, RateLimiterConfig};

#[tokio::main]
async fn main() {
    let limiter = RateLimiter::new(RateLimiterConfig {
        requests_per_second: 100,
        burst: 200,
    })
    .unwrap();

    let decision = limiter.check("client-id");
    if decision.permitted {
        // Process request
    } else {
        // Rate limit exceeded, retry after decision.retry_after
    }
}

Re-exports

pub use algorithm::Algorithm;

Modules

algorithm

Rate limiting algorithms.

middlewaremiddleware

Axum middleware integration for rate limiting.

tonic_middlewaretonic-support

Tonic middleware integration for gRPC rate limiting.

Structs

RateLimitDecision

Result of a rate limit check.

RateLimiter

A rate limiter that tracks requests per key and enforces limits.

RateLimiterBuilder

Builder for creating a rate limiter with a fluent API.

RateLimiterConfig

Configuration for creating a rate limiter.

Enums

Error

Error types that can occur during rate limiting operations.

Type Aliases

Result

A specialized Result type for rate limiting operations.