Struct TokenBucket 

pub struct TokenBucket { /* private fields */ }

Token bucket rate limiting algorithm.

The token bucket algorithm maintains a bucket of tokens that refills at a constant rate. Each request consumes one token. When the bucket is empty, requests are denied.

This implementation uses lock-free atomic operations for token accounting and flurry’s lock-free concurrent hashmap for per-key state management, achieving 10M+ operations per second with minimal contention.

Algorithm Details

• Capacity: Maximum number of tokens (burst size)
• Refill Rate: Tokens added per second
• Token Refill: Calculated based on elapsed time since last refill
• Concurrency:
  • Token updates use lock-free atomic compare-and-swap operations
  • Key lookup uses flurry’s lock-free HashMap with internal auto-tuning

Performance Characteristics

• Single-threaded: 17.8M ops/sec (+13% vs DashMap)
• Multi-threaded: 13.1M ops/sec at 2 threads (+26% vs DashMap)
• Multi-threaded: 11.6M ops/sec at 4 threads (+30% vs DashMap)
• Memory: O(number of unique keys) - see note on memory management

Memory Management

WARNING: This implementation creates a token bucket entry for each unique key and never removes them. For high-cardinality keys (e.g., per-request IDs), this can lead to unbounded memory growth. Consider:

• Using TTL-based eviction (future feature)
• Implementing periodic cleanup of idle entries
• Using lower-cardinality keys (e.g., per-user instead of per-request)

Implementations

impl TokenBucket

pub fn new(capacity: u64, refill_rate_per_second: u64) -> Self

Creates a new token bucket with the specified capacity and refill rate.

Arguments

• capacity - Maximum number of tokens (burst size), clamped to MAX_BURST if exceeded
• refill_rate_per_second - Number of tokens to add per second, clamped to MAX_RATE_PER_SEC if exceeded

Bounds

• Maximum capacity: 9,223,372,036,854,774 tokens (u64::MAX / 2000)
• Maximum refill rate: 9,223,372,036,854,774 tokens/sec

Values exceeding these bounds will be clamped to prevent arithmetic overflow.

Memory Management

By default, this does NOT evict idle keys. For high-cardinality use cases, use with_ttl() to enable automatic cleanup of idle entries.

Performance

Uses 256 sharded FlurryHashMaps for 256x reduced contention compared to a single HashMap. Each shard uses lock-free operations for concurrent access with automatic internal optimization. This architecture enables near-linear multi-threaded scaling.

Examples

use tokio_rate_limit::algorithm::TokenBucket;

// 100 requests per second with burst of 200
let bucket = TokenBucket::new(200, 100);

pub fn with_shard_count( capacity: u64, refill_rate_per_second: u64, _num_shards: usize, ) -> Self

👎Deprecated since 0.2.0: flurry uses internal auto-tuning; use new() instead

Creates a new token bucket with a custom shard count.

DEPRECATED: This method is deprecated as flurry uses internal auto-tuning and does not expose shard configuration. This method now simply calls new(). It is kept for backward compatibility but will be removed in a future version.

Use new() instead, which provides automatic optimization.

Arguments

• capacity - Maximum number of tokens (burst size)
• refill_rate_per_second - Number of tokens to add per second
• num_shards - Ignored (kept for backward compatibility)

Examples

use tokio_rate_limit::algorithm::TokenBucket;

// Use new() instead - provides automatic optimization
let bucket = TokenBucket::new(200, 100);

pub fn with_ttl( capacity: u64, refill_rate_per_second: u64, idle_ttl: Duration, ) -> Self

Creates a new token bucket with TTL-based eviction.

Keys that haven’t been accessed for idle_ttl duration will be removed during cleanup passes (triggered probabilistically on each check() call).

Arguments

• capacity - Maximum number of tokens (burst size)
• refill_rate_per_second - Number of tokens to add per second
• idle_ttl - Duration after which idle keys are evicted

Examples

use tokio_rate_limit::algorithm::TokenBucket;
use std::time::Duration;

// Evict keys idle for more than 1 hour
let bucket = TokenBucket::with_ttl(200, 100, Duration::from_secs(3600));

pub fn with_ttl_and_shard_count( capacity: u64, refill_rate_per_second: u64, idle_ttl: Duration, _num_shards: usize, ) -> Self

👎Deprecated since 0.2.0: flurry uses internal auto-tuning; use with_ttl() instead

Creates a new token bucket with both TTL-based eviction and custom shard count.

DEPRECATED: This method is deprecated as flurry uses internal auto-tuning. This method now simply calls with_ttl(). It is kept for backward compatibility but will be removed in a future version.

Use with_ttl() instead, which provides automatic optimization.

Arguments

• capacity - Maximum number of tokens (burst size)
• refill_rate_per_second - Number of tokens to add per second
• idle_ttl - Duration after which idle keys are evicted
• num_shards - Ignored (kept for backward compatibility)

Examples

use tokio_rate_limit::algorithm::TokenBucket;
use std::time::Duration;

// Use with_ttl() instead - provides automatic optimization
let bucket = TokenBucket::with_ttl(200, 100, Duration::from_secs(3600));

Trait Implementations

impl Algorithm for TokenBucket

fn check<'life0, 'life1, 'async_trait>( &'life0 self, key: &'life1 str, ) -> Pin<Box<dyn Future<Output = Result<RateLimitDecision>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Checks if a request for the given key should be permitted.

fn check_with_cost<'life0, 'life1, 'async_trait>( &'life0 self, key: &'life1 str, cost: u64, ) -> Pin<Box<dyn Future<Output = Result<RateLimitDecision>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Checks if a request with the given cost should be permitted.