Skip to main content

Module rate_limit

Module rate_limit 

Source
Expand description

Token bucket rate limiter for cloud API calls.

Prevents hitting provider rate limits (IBM: 5 jobs/min, AWS: 10 req/s, etc.) by tracking token consumption per backend. The token bucket algorithm smooths bursty traffic while respecting sustained-rate limits.

Structsยง

RateLimiter
Per-provider token-bucket rate limiter.
TokenBucket
A token bucket: accumulates capacity tokens at refill_rate tokens/second.