mettle

A resilience toolkit for Rust.

Composable, testable primitives for handling failure. Documentation.

Install

cargo add mettle

Blocking only, without an async runtime (no tokio):

cargo add mettle --no-default-features --features blocking

Example

use mettle::retry;
use std::time::Duration;

// Retry with sensible defaults (exponential backoff, up to 3 retries),
// then override only what you need.
let body = retry(|| async { fetch(&url).await })
    .when(|e: &FetchError| e.is_transient())   // skip permanent errors
    .max_elapsed(Duration::from_secs(30))       // give up after ~30s total
    .await?;

No async runtime? The blocking twin is identical but ends in .call() instead of .await.

Retrying on a fixed schedule means every client that failed together retries together, so a service that is coming back up gets a synchronized wave. Jitter spreads them out:

use mettle::{Backoff, DecorrelatedBackoff, DecorrelatedBackoffConfig, ExponentialBackoff};

// Randomize any strategy's delays into 0 ..= delay ("full jitter")...
let backoff = ExponentialBackoff::default().jittered();

// ...or use decorrelated jitter, where each delay is drawn from the previous one.
let backoff = DecorrelatedBackoff::new(DecorrelatedBackoffConfig::default())?;

Which one: .jittered() works on any strategy, including one you wrote, and spreads delays as widely as possible. DecorrelatedBackoff is its own strategy and never draws below its base, so reach for it when you want a floor under every wait. The trade is that a floor also means never retrying sooner than base, so a dependency that frees up early isn't picked up until then.

Both seed from entropy by default and take a fixed seed (with_seed) when you want a test to replay the same delays.

When a retry gives up you get a RetryError, which says what stopped it:

match retry(|| async { fetch(&url).await }).await {
    Ok(body) => body,
    Err(e) => {
        // "max_elapsed after 5 attempts in 29.4s: connection refused"
        tracing::error!("{} after {} attempts in {:?}: {}",
            e.stop_reason().as_str(), e.attempts(), e.elapsed(), e.error());
        return Err(e.into());   // ?-able into Box<dyn Error> / anyhow
    }
}

Only want the underlying error? .map_err(RetryError::into_error).

Tools

Each tool comes with a runnable example. Start there:

retry: async examples/retry.rs · blocking examples/blocking_retry.rs

Retries emit tracing events out of the box (target mettle::retry). Install any subscriber (e.g. tracing_subscriber::fmt::init()) to see them.

Status

v0.x, with async (Tokio) and blocking APIs. Expect breaking changes before 1.0.

License

Licensed under either of

at your option.