Crate tower_resilience_coalesce 

Request coalescing for Tower services.

This crate provides a Tower layer that coalesces (deduplicates) concurrent identical requests, ensuring only one request executes while others wait for its result. This prevents “cache stampede” or “thundering herd” problems.

How It Works

1. The first request with a given key begins execution
2. Subsequent requests with the same key wait for the first to complete
3. All waiting requests receive a clone of the result
4. Errors are also propagated to all waiters

Example

use tower_resilience_coalesce::CoalesceLayer;
use tower::{Service, ServiceBuilder, ServiceExt};

let service = ServiceBuilder::new()
    .layer(CoalesceLayer::new(|req: &Request| req.id.clone()))
    .service(backend);

Use Cases

• Cache refresh protection: When a cached value expires, multiple requests may try to refresh it simultaneously. Coalescing ensures only one refresh happens.

• Expensive computations: Deduplicate requests for the same expensive operation (e.g., report generation, ML inference).

• Rate-limited APIs: Reduce calls to external APIs that have rate limits by coalescing identical requests.

• Database queries: Combine identical queries that arrive within a short window to reduce database load.

Requirements

• The key type must implement Hash + Eq + Clone + Send + Sync
• The response type must implement Clone
• The error type must implement Clone

Prior Art

This pattern is also known as:

• Singleflight (Go’s golang.org/x/sync/singleflight)
• Request deduplication
• Request collapsing

Structs

CoalesceConfig

Configuration for the coalesce layer.

CoalesceConfigBuilder

Builder for coalesce configuration.

CoalesceLayer

A Tower layer that coalesces concurrent identical requests.

CoalesceService

A service that coalesces concurrent identical requests.

Enums

CoalesceError

Error type for coalesced requests.

CoalesceFuture

Future for coalesced requests.