Hedging middleware for Tower services.

Hedging reduces tail latency by executing parallel redundant requests. Instead of waiting for a slow request to complete, hedging fires additional requests after a configurable delay and returns whichever completes first.

Overview

The hedging pattern is useful when:

Tail latency (P99/P999) is critical
Operations are idempotent and safe to retry
You can trade increased resource usage for lower latency

Presets

use tower_resilience_hedge::HedgeLayer;

let conservative = HedgeLayer::conservative(); // 500ms delay, 2 attempts
let standard = HedgeLayer::standard();         // 100ms delay, 3 attempts
let aggressive = HedgeLayer::aggressive();     // 50ms delay, 5 attempts

Modes

Latency Mode (delay > 0)

Wait a specified duration before firing hedge requests. This is the default and most common mode - it only sends extra requests if the primary is slow.

use tower_resilience_hedge::HedgeLayer;
use std::time::Duration;

// No type parameters needed! Fire a hedge request if primary takes > 100ms
let layer = HedgeLayer::builder()
    .delay(Duration::from_millis(100))
    .max_hedged_attempts(2)
    .build();

Parallel Mode (delay = 0)

Fire all requests simultaneously and return the fastest response. Use when latency is critical and you can afford the resource cost.

use tower_resilience_hedge::HedgeLayer;

// No type parameters needed! Fire 3 requests immediately, return fastest
let layer = HedgeLayer::builder()
    .no_delay()
    .max_hedged_attempts(3)
    .build();

Example

use tower::{Service, ServiceExt, Layer};
use tower_resilience_hedge::HedgeLayer;
use std::time::Duration;

// Define a simple cloneable error type
#[derive(Clone, Debug)]
struct MyError;
impl std::fmt::Display for MyError {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        write!(f, "MyError")
    }
}
impl std::error::Error for MyError {}

# async fn example() -> Result<(), Box<dyn std::error::Error>> {
// Create a service that sometimes responds slowly
let service = tower::service_fn(|req: String| async move {
    // Simulate variable latency
    Ok::<_, MyError>(format!("response: {}", req))
});

// Wrap with hedging - fire hedge after 50ms (no type parameters needed!)
let hedge = HedgeLayer::builder()
    .delay(Duration::from_millis(50))
    .max_hedged_attempts(2)
    .build();

let mut service = hedge.layer(service);

let response = service.ready().await?.call("hello".to_string()).await?;
println!("Got response: {}", response);
# Ok(())
# }

Cancellation

When one request succeeds, all other in-flight requests are cancelled by dropping their futures. This relies on the inner service supporting cooperative cancellation.

Type Requirements

Hedging has specific trait bounds that differ from other resilience patterns:

Req: Clone - Required because the request is cloned to send parallel requests. Each hedge attempt needs its own copy of the request.
E: Clone - Required for error handling. When multiple attempts fail, errors need to be collected and stored to return the final error.

If your request or error types don't implement Clone, consider:

Wrapping them in Arc (e.g., Arc<MyRequest>)
Using a different resilience pattern like Retry which doesn't require cloning requests

tower-resilience-hedge 0.9.3