quota

A high-performance in-memory rate limiter for Rust, completely lock-free.

It uses a mix of Leaky Token Bucket & GCRA.

Scales with a compact 16 bytes to 29 bytes of memory per quota

• Quota size: 8 bytes (*)
• Last Access Time: 8 bytes (*)
• Last Refill Time: 8 bytes (Enabled by default for an efficient per-quota clock)
• Generation & Occupied: 5 bytes (Enabled by default for preventing dangling references from PoolQuota::quota())

The library uses SoA architecture, so the numbers above are accurate, no byte-alignment involved, which can make it efficient for SIMD processing in the future.

We provide 3 essential clean primitives: Quota, QuotaPolicy, and finally QuotaPool that combines both of them.

Example use of the simple Quota (A simple 8-byte number in memory):

use quota::Quota;

fn main() {
    let quota = Quota::with_initial_tokens(10);

    let mut results = vec![];
    for _ in 0..100 {
        results.push(quota.consume(1));
    }

    assert_eq!(results.iter().filter(|r| r.is_ok()).count(), 10);
    assert_eq!(results.iter().filter(|r| r.is_err()).count(), 90);
}

Example use of applying QuotaPolicy with a maximum capacity and RefillRate:

use quota::{Quota, QuotaPolicy, RefillRate};

fn main() {
    let policy = QuotaPolicy::new()
        .set_capacity(10.0) // Maximum Capacity to apply to a Quota per tick
        .set_refill_rate(RefillRate::per_micro(100.0)); // Refill Rate to apply to a Quota per tick

    let mut quota = Quota::with_initial_tokens(10);

    let mut results = vec![];
    for _ in 0..100 {
        policy.tick(1, &mut quota);
        results.push(quota.consume(1));
    }

    assert_eq!(results.iter().filter(|r| r.is_ok()).count(), 19);
    assert_eq!(results.iter().filter(|r| r.is_err()).count(), 81);
}

And now the main QuotaPool:

use quota::{RefillRate, QuotaPolicy, QuotaPool};

fn main() {
    let policy = QuotaPolicy::new()
        .set_capacity(10.0)
        .set_refill_rate(RefillRate::per_sec(3));
    // TODO:
    // We'll later add a custom tick interval rather than
    // ticking every time per ::consume, which is useful if you want
    // the rate limit to refill (the term would be "reset" if you always refill the Quota entirely)
    // every N interval for prolonged blocked access (Discrete refilling)
    // rather than refilling the Quota all the time. (Continuous refilling)
    // Examples:
    // - Refill Rate of 3600 tokens per hour & Tick Interval of 1 hour means the 3600 tokens are added every hour
    // - Refill Rate of 3600 tokens per hour & Tick Interval of 30 minutes means 1800 tokens are added every 30 minutes

    let pool = QuotaPool::new(policy, 10);

    let mut results = vec![];
    for _ in 0..100 {
        results.push(pool.consume("testing", 1));
    }
}