Crate tower_resilience_adaptive 

Adaptive concurrency limiter for Tower services.

This crate provides a Tower layer that dynamically adjusts concurrency limits based on observed latency and error rates, using algorithms like AIMD or Vegas.

Unlike static concurrency limits which require manual tuning, adaptive limiters automatically find the optimal concurrency for your downstream services.

Algorithms

AIMD (Additive Increase Multiplicative Decrease)

The classic TCP congestion control algorithm:

• On success with low latency: increase limit by a fixed amount
• On failure or high latency: decrease limit by a factor (e.g., halve it)

This creates a “sawtooth” pattern as it continuously probes for capacity.

Vegas

A more sophisticated algorithm that uses RTT measurements:

• Estimates queue depth from RTT variations
• Increases limit when queue is small (under-utilized)
• Decreases limit when queue is large (congested)

Vegas is more stable than AIMD and avoids the sawtooth pattern.

Example

use tower_resilience_adaptive::{AdaptiveLimiterLayer, Aimd};
use tower::{Service, ServiceBuilder, ServiceExt};
use std::time::Duration;

// Create a service
let service = tower::service_fn(|req: String| async move {
    Ok::<_, std::convert::Infallible>(format!("Hello, {}!", req))
});

// Wrap with adaptive limiter using AIMD
let mut service = ServiceBuilder::new()
    .layer(AdaptiveLimiterLayer::new(
        Aimd::builder()
            .initial_limit(10)
            .min_limit(1)
            .max_limit(100)
            .latency_threshold(Duration::from_millis(100))
            .build()
    ))
    .service(service);

// The limit will automatically adjust based on response latency
let response = service.ready().await?.call("World".to_string()).await?;

Using Vegas Algorithm

use tower_resilience_adaptive::{AdaptiveLimiterLayer, Vegas};
use tower::ServiceBuilder;

let layer = AdaptiveLimiterLayer::new(
    Vegas::builder()
        .initial_limit(10)
        .alpha(3)  // Increase when queue < 3
        .beta(6)   // Decrease when queue > 6
        .build()
);

Combining with Other Patterns

The adaptive limiter works well with other resilience patterns:

use tower_resilience_adaptive::{AdaptiveLimiterLayer, Aimd};
use tower_resilience_retry::RetryLayer;
use tower_resilience_circuitbreaker::CircuitBreakerLayer;
use tower::ServiceBuilder;

let service = ServiceBuilder::new()
    // Outer: circuit breaker for catastrophic failures
    .layer(CircuitBreakerLayer::builder().build())
    // Middle: adaptive concurrency limiting
    .layer(AdaptiveLimiterLayer::new(Aimd::builder().build()))
    // Inner: retry transient failures
    .layer(RetryLayer::builder().max_attempts(3).build())
    .service(my_service);

Prior Art

This implementation is inspired by:

• Netflix concurrency-limits
• Uber Cinnamon
• Vector Adaptive Request Concurrency

Structs

AdaptiveFuture

Future returned by AdaptiveService.

AdaptiveLimiterLayer

A Tower layer that applies adaptive concurrency limiting.

AdaptiveLimiterLayerBuilder

Builder for configuring an adaptive limiter layer.

AdaptiveService

A service that applies adaptive concurrency limiting.

Aimd

AIMD (Additive Increase Multiplicative Decrease) algorithm.

AimdBuilder

Builder for AIMD algorithm.

Vegas

TCP Vegas algorithm for concurrency control.

VegasBuilder

Builder for Vegas algorithm.

Enums

AdaptiveError

Error type for adaptive limiter.

Algorithm

Algorithm selection enum for the adaptive limiter.

Traits

ConcurrencyAlgorithm

Trait for adaptive concurrency control algorithms.

IntoLayer

Extension trait for building layers from algorithm builders.