Struct ConcurrencyConfig 

pub struct ConcurrencyConfig {
    pub max_concurrent_requests: usize,
    pub requests_per_minute: usize,
    pub enabled: bool,
    pub semaphore_enabled: bool,
}

Concurrency control configuration.

This controls how LLM requests are rate-limited and throttled to avoid overwhelming the API.

Fields

max_concurrent_requests: usize

Maximum concurrent LLM API calls.

This limits how many requests can be in-flight at the same time. Default: 10

requests_per_minute: usize

Rate limit: requests per minute.

This is a soft limit using token bucket algorithm. Default: 500 (OpenAI default tier)

enabled: bool

Alias for enabled - whether rate limiting is enabled.

When disabled, only semaphore-based concurrency control is used. Default: true

semaphore_enabled: bool

Whether to enable concurrency limiting via semaphore.

When disabled, only rate limiting is used. Default: true

Implementations

impl ConcurrencyConfig

pub fn new() -> Self

Create a new config with defaults.

pub fn with_max_concurrent_requests(self, max: usize) -> Self

Set the maximum concurrent requests.

pub fn with_requests_per_minute(self, rpm: usize) -> Self

Set the requests per minute rate limit.

pub fn with_enabled(self, enabled: bool) -> Self

Enable or disable rate limiting.

pub fn high_throughput() -> Self

Create a config for high-throughput scenarios.

Uses higher limits suitable for paid API tiers.

pub fn conservative() -> Self

Create a config for conservative scenarios.

Uses lower limits to avoid rate limit errors.

pub fn unlimited() -> Self

Create a config that disables all limits.

Useful for testing or when external rate limiting is used.

Trait Implementations

impl Clone for ConcurrencyConfig

fn clone(&self) -> ConcurrencyConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ConcurrencyConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for ConcurrencyConfig

fn default() -> Self

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for ConcurrencyConfig

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl From<ConcurrencyConfig> for ConcurrencyConfig

fn from(config: ConcurrencyConfig) -> Self

Converts to this type from the input type.

impl Serialize for ConcurrencyConfig

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.