Skip to main content

Module flight

Module flight 

Source
Expand description

Cold-miss single-flight: dedup concurrent fills of the same key. Cold-miss single-flight: dedup concurrent fills of the same key.

Under metered-misses pricing a stampede is literally billable — N tasks missing the same key at once means N backend misses and N executions of the wrapped function. CacheKit::single_flight collapses that to one:

  • In-process (always available): a per-key async mutex. The first task through becomes the leader and computes; concurrent tasks queue behind it and re-check the cache once the leader finishes.
  • Cross-process (reliability feature, native, backend implements LockableBackend — CachekitIO and Redis do): the leader additionally takes a distributed fill lock. If another process already holds it, this process polls the cache for the other side’s fill instead of recomputing, and computes anyway once the poll budget is exhausted (fail-open — a stampede beats unavailability).

The #[cachekit] macro wires this in automatically around its miss path. Manual usage follows the same shape:

if let Some(_v) = cache.get::<String>("expensive").await? {
    return Ok(());
}
let mut flight = cache.single_flight("expensive").await;
while flight.wait_for_fill().await {
    if let Some(_v) = cache.get::<String>("expensive").await? {
        flight.release().await; // another worker filled it
        return Ok(());
    }
}
let value = "computed".to_owned(); // expensive work — runs once
cache.set("expensive", &value).await?;
flight.release().await;

Structs§

SingleFlight
Guard for a single-flight fill, returned by CacheKit::single_flight.