load-balancer

A set of asynchronous load balancers for Rust, built on Tokio.

All balancers (except RateLimiter) implement the LoadBalancer<T> trait:

pub trait LoadBalancer<T>: Send + Sync + Clone + 'static {
    fn alloc(&self) -> impl Future<Output = T> + Send;
    fn try_alloc(&self) -> Option<T>;
}

Examples

RoundRobin

use load_balancer::round_robin::RoundRobin;
use load_balancer::LoadBalancer;

#[tokio::main]
async fn main() {
    let lb = RoundRobin::new(vec!["node-a", "node-b", "node-c"]);

    for _ in 0..6 {
        println!("{}", lb.alloc().await);
    }
}

node-a
node-b
node-c
node-a
node-b
node-c

Random

use load_balancer::random::Random;
use load_balancer::LoadBalancer;

#[tokio::main]
async fn main() {
    let lb = Random::new(vec!["s1", "s2", "s3", "s4"]);

    for _ in 0..5 {
        println!("{}", lb.alloc().await);
    }
}

s3
s1
s4
s2
s3

Window

Limits the number of allocations per entry within a time window.

use load_balancer::window::Window;
use load_balancer::LoadBalancer;
use std::time::Duration;

#[tokio::main]
async fn main() {
    // node-1: max 5/sec, node-2: max 3/sec
    let lb = Window::new(vec![(5, "node-1"), (3, "node-2")]);

    for _ in 0..10 {
        println!("{}", lb.alloc().await);
    }
}

node-1
node-1
node-1
node-1
node-1
node-2
node-2
node-2
node-1   ← window reset after 1s
node-1

Custom interval:

let lb = Window::new_interval(
    vec![(3, "db-1"), (1, "db-2")],
    Duration::from_secs(5),
);

FaultTolerant

Extends Window with error tracking: entries that exceed their error threshold are auto-disabled until the next reset cycle. Call success() to gradually restore an entry's reputation.

use load_balancer::fault_tolerant::FaultTolerant;
use load_balancer::LoadBalancer;

#[tokio::main]
async fn main() {
    // (max_count, max_error_count, value)
    let lb = FaultTolerant::new(vec![
        (5, 2, "api-1"),
        (5, 2, "api-2"),
        (5, 2, "api-3"),
    ]);

    for _ in 0..10 {
        let (index, node) = lb.alloc_skip(usize::MAX).await;

        // Simulate: the first node always fails
        if index == 0 {
            lb.failure(index);
            println!("{node} → FAIL (errors will disable it)");
        } else {
            lb.success(index);
            println!("{node} → OK");
        }
    }
}

api-1 → FAIL
api-1 → FAIL
api-2 → OK    ← api-1 is now disabled
api-2 → OK
api-2 → OK
api-2 → OK
api-2 → OK
api-3 → OK
api-3 → OK
api-3 → OK

Cooldown

Each entry enters a cooldown period after allocation and cannot be reused until it expires. Use tokio::time::sleep to wait it out, or let alloc().await block until an entry becomes available.

use load_balancer::cooldown::Cooldown;
use load_balancer::LoadBalancer;
use std::time::Duration;

#[tokio::main]
async fn main() {
    let lb = Cooldown::new(vec![
        (Duration::from_secs(2), "slow"),
        (Duration::ZERO, "fast"),
    ]);

    // slow gets picked first, then enters its 2 s cooldown
    println!("{}", lb.alloc().await); // slow
    println!("{}", lb.alloc().await); // fast (slow is cooling down)
    println!("{}", lb.alloc().await); // fast

    // wait for slow's cooldown to expire
    tokio::time::sleep(Duration::from_secs(2)).await;

    println!("{}", lb.alloc().await); // slow again
    println!("{}", lb.alloc().await); // fast
}

slow
fast
fast
slow
fast

ProxyPool

HTTP proxy pool with health checking, latency-based sorting, and automatic dead-proxy removal.

use load_balancer::proxy_pool::ProxyPool;
use load_balancer::LoadBalancer;
use std::time::Duration;

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let pool = ProxyPool::new([
        "socks5://127.0.0.1:1080",
        "http://proxy1.example.com:8080",
        "http://proxy2.example.com:8080",
    ])
    .timeout(Duration::from_secs(10))
    .max_check_concurrency(500);

    // Health check: remove dead proxies, sort by latency ascending
    pool.check(2).await?;

    // Auto-check every 60 seconds
    pool.spawn_check(Duration::from_secs(60), 2).await?;

    // Round-robin allocation
    for _ in 0..3 {
        println!("{}", pool.alloc().await);
    }

    Ok(())
}

Adding proxies dynamically:

pool.extend(["http://new-proxy:3128"]).await;           // append only, no validation
pool.extend_check(["http://new-proxy:3128"], 2).await?; // append and validate immediately

RateLimiter

A standalone global rate limiter. Does not implement LoadBalancer; useful for protecting downstream APIs.

use load_balancer::rate_limit::RateLimiter;
use std::time::Duration;

#[tokio::main]
async fn main() {
    // Max 3 requests per second
    let limiter = RateLimiter::new_rate(3);

    for i in 0..5 {
        if limiter.check().await {
            println!("request {i} → allowed");
        } else {
            let wait = limiter.time_until_available().await;
            println!("request {i} → denied, retry in {wait:?}");
        }
    }
}

request 0 → allowed
request 1 → allowed
request 2 → allowed
request 3 → denied, retry in 1s
request 4 → denied, retry in 1s

RateLimiterMap

Per-key rate limiting with independent limits per key. Keys that haven't been inserted are rejected by default — call [insert_rate] or [insert_rate_interval] first.

use load_balancer::rate_limit::RateLimiterMap;
use std::time::Duration;

#[tokio::main]
async fn main() {
    let map = RateLimiterMap::new();

    // Each key gets its own limit
    map.insert_rate("alice", 3);                                // 3 req/s
    map.insert_rate_interval("bob", 10, Duration::from_secs(5)); // 10 req/5s

    for i in 0..4 {
        if map.check(&"alice").await {
            println!("alice {i} → allowed");
        } else {
            println!("alice {i} → denied");
        }
    }

    // Not inserted — rejected
    assert!(!map.check(&"charlie").await);

    // Remove a key to lift its limit (removed keys are also rejected)
    map.remove(&"alice");
}

alice 0 → allowed
alice 1 → allowed
alice 2 → allowed
alice 3 → denied

Custom Timers

Every balancer (RoundRobin, Random, Cooldown, Window, FaultTolerant, ProxyPool) exposes an update_timer method that spawns a background task to run a custom handler at a fixed interval. This is useful for dynamic reconfiguration, reloading entries from a file, or periodic health checks without blocking.

use load_balancer::round_robin::RoundRobin;
use std::time::Duration;

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let lb = RoundRobin::new(vec!["node-a", "node-b"]);

    // Refresh the pool every 30 seconds
    let handle = lb.update_timer(
        |inner| async move {
            // e.g. fetch fresh node list, replace entries, reset cursor
            println!("reloading…");
            Ok(())
        },
        Duration::from_secs(30),
    ).await?;

    // … use the balancer …

    handle.abort(); // stop the timer when done
    Ok(())
}