Struct ModelRateLimit 

pub struct ModelRateLimit { /* private fields */ }

Per-model rate / concurrency limits attached to one router slot.

Construct with ModelRateLimit::per_minute (or ModelRateLimit::per_second) and tighten the optional dimensions with the with_* builders. Pass to RouterLLM::with_model_rate_limited or RouterLLM::with_last_rate_limit.

Implementations

impl ModelRateLimit

pub fn per_minute(requests_per_minute: usize) -> Self

At most requests_per_minute admitted per 60-second window, unlimited concurrency, unbounded queue, 60-second wait timeout.

pub fn per_second(requests_per_second: usize) -> Self

At most requests_per_second admitted per one-second window.

pub fn with_window(self, window: Duration) -> Self

Overrides the rate window length (e.g. a provider quota stated per day). The wait timeout is intentionally left untouched: a queued caller may legitimately wait several windows deep, so configure ModelRateLimit::with_wait_timeout separately when shortening.

pub fn with_max_concurrent(self, max_concurrent: usize) -> Self

Bounds in-flight calls. The slot stays occupied until the whole response (including a streamed body) has finished. 0 (the default) disables the concurrency dimension.

pub fn with_max_queue(self, max_queue: usize) -> Self

Bounds how many callers may wait for admission; 0 (the default) means an unbounded FIFO queue. A full queue rejects immediately with RouterError::RateLimited so the router can fall through to a fallback model without delay.

pub fn with_wait_timeout(self, wait_timeout: Duration) -> Self

Caps how long one caller waits before the router gives up on this slot and tries the next model. Defaults to one rate window.

pub fn requests_per_window(&self) -> usize

Configured requests-per-window.

pub fn window(&self) -> Duration

Configured rate window.

pub fn max_concurrent(&self) -> usize

Configured concurrency cap (0 = unlimited).

pub fn max_queue(&self) -> usize

Configured queue depth (0 = unbounded).

pub fn wait_timeout(&self) -> Duration

Configured admission wait timeout.

Trait Implementations

impl Clone for ModelRateLimit

fn clone(&self) -> ModelRateLimit

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ModelRateLimit

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.