Expand description
Cold-miss single-flight: dedup concurrent fills of the same key. Cold-miss single-flight: dedup concurrent fills of the same key.
Under metered-misses pricing a stampede is literally billable — N tasks
missing the same key at once means N backend misses and N executions of
the wrapped function. CacheKit::single_flight
collapses that to one:
- In-process (always available): a per-key async mutex. The first task through becomes the leader and computes; concurrent tasks queue behind it and re-check the cache once the leader finishes.
- Cross-process (
reliabilityfeature, native, backend implementsLockableBackend— CachekitIO and Redis do): the leader additionally takes a distributed fill lock. If another process already holds it, this process polls the cache for the other side’s fill instead of recomputing, and computes anyway once the poll budget is exhausted (fail-open — a stampede beats unavailability).
The #[cachekit] macro wires this in automatically around its miss path.
Manual usage follows the same shape:
if let Some(_v) = cache.get::<String>("expensive").await? {
return Ok(());
}
let mut flight = cache.single_flight("expensive").await;
while flight.wait_for_fill().await {
if let Some(_v) = cache.get::<String>("expensive").await? {
flight.release().await; // another worker filled it
return Ok(());
}
}
let value = "computed".to_owned(); // expensive work — runs once
cache.set("expensive", &value).await?;
flight.release().await;Structs§
- Single
Flight - Guard for a single-flight fill, returned by
CacheKit::single_flight.