Skip to main content

Module rate_limiter

Module rate_limiter 

Source
Expand description

Simple in‑process rate limiter for tool execution.

This is a lightweight fallback used when the full governor crate is not available in the build. It limits the number of tool calls per second globally for the process. The limiter is deliberately cheap – a Mutex protecting a counter and a timestamp – because the surrounding code already performs async work, so contention is minimal.

The implementation is intentionally minimalistic: it provides a RateLimiter struct with a try_acquire() method that returns Ok(()) when the call is allowed or an Err when the limit would be exceeded. Callers can decide how to handle the error (e.g., retry after a delay or surface a user‑friendly message).

Usage example:

use vtcode_core::tools::rate_limiter::GLOBAL_RATE_LIMITER;

fn execute_tool() -> anyhow::Result<()> {
    GLOBAL_RATE_LIMITER.try_acquire()?;
    // … actual tool logic …
    Ok(())
}

The limiter is configured via environment variables to keep the core library free of additional runtime configuration files:

  • VTTOOL_RATE_LIMIT – maximum calls per second (default = 20).
  • VTTOOL_BURST – maximum burst size (default = rate limit, min 5).

The values are read once at startup.

This file is added as part of the optimization plan to provide a central rate‑limiting mechanism for all external tool invocations (PTY, web fetch, filesystem, etc.).

Structs§

PerToolRateLimiter
Per-tool rate limiter for finer-grained control. Each tool gets its own token bucket, allowing different rate limits per tool.
RateLimiterConfig
Configuration for the limiter.
RateLimiterInner
Simple token‑bucket implementation.

Statics§

GLOBAL_RATE_LIMITER
PER_TOOL_RATE_LIMITER
Global per-tool rate limiter instance.

Functions§

try_acquire
Public API – try to acquire permission for a tool call.
try_acquire_for
Try to acquire permission for a specific tool. Uses per-tool rate limiting for finer-grained control.

Type Aliases§

RateLimiter
Public alias for benchmark compatibility