Crate prob_rate_limiter

prob-rate-limiter

ProbRateLimiter is a probabilistic rate limiter. When load approaches the configured limit, the struct chooses randomly whether to accept or reject each request. It adjusts the probability of rejection so throughput is steady around the limit.

Use Cases

• Shed load to prevent overload
• Avoid overloading the services you depend on
• Control costs

Features

• Tiny, uses 44 bytes
• 100% test coverage
• Optimized: 32ns per check, 31M checks per second on an i5-8259U
• No unsafe or unsafe deps

Limitations

• Requires a mutable reference.
• Not fair. Treats all requests equally, regardless of source. A client that overloads the server will consume most of the throughput.

Alternatives

• r8limit
  • Uses a sliding window
  • No unsafe or deps
  • Optimized: 48ns per check, 21M checks per second on an i5-8259U
  • Requires a mutable reference.
• governor
  • Uses a non-mutable reference, easy to share between threads
  • Popular
  • Good docs
  • Optimized: 29ns per check on an i5-8259U.
  • Unnecessary unsafe
  • Uses non-standard mutex library parking_lot
  • Uses a complicated algorithm
• leaky-bucket
  • Async tasks can wait for their turn to use a resource.
  • Unsuitable for load shedding because there is no try_acquire.

Related Crates

• safe-dns uses this

Example

let mut limiter = ProbRateLimiter::new(10.0).unwrap();
let mut now = Instant::now();
assert!(limiter.check(5, now));
assert!(limiter.check(5, now));
now += Duration::from_secs(1);
assert!(limiter.check(5, now));
assert!(limiter.check(5, now));
now += Duration::from_secs(1);
assert!(limiter.check(5, now));
assert!(limiter.check(5, now));
now += Duration::from_secs(1);
assert!(limiter.check(5, now));
assert!(limiter.check(5, now));
now += Duration::from_secs(1);
assert!(limiter.check(5, now));
assert!(limiter.check(5, now));
assert!(!limiter.check(5, now));

Cargo Geiger Safety Report

Changelog

• v0.1.1 - Simplify new. Add more docs.
• v0.1.0 - Initial version

TO DO

• Publish
• Add graph from the benchmark.

Structs

ProbRateLimiter

A probabilistic rate-limiter.