Struct ClockTtlCache 

pub struct ClockTtlCache<K, V>
where
    K: Hash + Eq + Send + Sync,
    V: Clone + Send + Sync,
{ /* private fields */ }

TTL+LRU cache with all time decisions routed through an injected Clock.

Capacity vs. TTL

Two independent eviction triggers:

• TTL: checked on read. Expired entries are removed lazily when touched; cleanup_expired sweeps the whole map for callers who want to reclaim memory eagerly.
• Capacity: enforced at insert time by LruCache. Inserting into a full cache evicts the least-recently-used entry.

Concurrency

Internal state is wrapped in a parking_lot::Mutex. Reads briefly acquire the lock to update LRU recency. The single mutex becomes the bottleneck only once multiple threads contend on the same hot key; benches/cache_bench.rs quantifies the regression. A sharded variant is deferred until real profiling shows it matters.

DST guarantees

Every TTL decision and every expires_at_micros calculation goes through clock.now(). Tests using MockClock can advance time in arbitrary jumps and observe deterministic eviction. There are no background tasks and no calls into Instant::now() / chrono::Utc::now().

Implementations

impl<K, V> ClockTtlCache<K, V>

where K: Hash + Eq + Clone + Send + Sync, V: Clone + Send + Sync,

pub fn new(capacity: NonZeroUsize, ttl: Duration, clock: Arc<dyn Clock>) -> Self

Construct a cache with the given capacity, TTL, and Clock.

pub fn stats(&self) -> CacheStats

Snapshot of the live observability counters (hits, misses, evictions, single-flight joins, …). See CacheStats for field semantics.

Cheap (atomic loads only); safe to call on the hot path or from a metrics scrape endpoint.

pub fn reset_stats(&self)

Reset every counter to zero. Useful for tests and for resetting a metrics window after a scrape; production telemetry pipelines typically just take rate-of-change of the monotonic counters and shouldn’t need this.

pub fn get(&self, key: &K) -> Option<V>

Look up a key. Returns None if absent or expired (and removes the expired entry as a side effect; lazy TTL sweep).

pub fn insert(&self, key: K, value: V)

Insert or replace a value with the configured TTL.

pub fn invalidate(&self, key: &K) -> bool

Remove a single entry. Also removes any in-flight load cell for the same key, so a concurrent get_or_try_insert_with load that resolves after this call cannot re-cache the now-invalidated value. Returns true if anything was removed (cache row, in-flight cell, or both).

Concurrency

Two separate critical sections: inflight first, then LRU. Lock-ordering rule (inflight before LRU) is preserved. A concurrent load whose post-resolve runs between our inflight removal and our LRU removal sees its cell missing from inflight and skips the LRU promotion (see get_or_try_insert_with’s Ok arm). A concurrent load whose post-resolve runs before our inflight removal already inserted into LRU, which our LRU pop then removes. Both paths converge on “key is not cached afterward.”

pub fn invalidate_by(&self, predicate: impl Fn(&K) -> bool) -> usize

Remove every cache entry whose key matches predicate. Also removes matching in-flight load cells so concurrent loads resolving after this call cannot re-cache invalidated values. Returns the number of cache entries removed (LRU entries only; in-flight removals are bookkeeping side effects).

O(n) over the cache plus O(m) over the in-flight map. Use for “evict everything for principal X” after a role change; small N for typical caches; if your cardinality is large enough that this is too slow, consider maintaining a secondary index outside this primitive.

pub fn invalidate_all(&self)

Drop every entry from the cache. Also drops every in-flight load cell, so any load resolving after this call is barred from promoting its value to the cache (post-invalidate_all cache state must be empty).

pub fn cleanup_expired(&self) -> usize

Walk the cache and remove every expired entry. Returns the number of entries reclaimed.

Optional: the cache evicts expired entries lazily on access via get. Call this if you want to reclaim memory ahead of next access (e.g. from a periodic Clock-aware scheduled task).

pub async fn get_or_try_insert_with<F, Fut, E>( &self, key: K, fetcher: F, ) -> Result<V, E>

where F: FnOnce() -> Fut, Fut: Future<Output = Result<V, E>>,

Load key from the cache, or run fetcher to populate it. Single-flight: concurrent calls for the same key share one fetcher invocation, and the rest await the in-flight cell.

Errors

On fetcher error the in-flight cell is removed so subsequent callers retry rather than re-await a known-failing future. The error is propagated to every concurrent caller that joined this in-flight load.

Panic safety

If fetcher panics, the in-flight cell is removed via an RAII guard whose Drop runs during panic-unwind. Subsequent callers for the same key get a fresh fetcher invocation rather than joining a permanently-uninitialised cell.

Interaction with concurrent invalidate

Invalidate wins against concurrent loads. If invalidate / invalidate_by / invalidate_all runs while a load is in flight for key, the loaded value is not promoted to the cache. The caller that triggered the in-flight load (and joiners) still receive the loaded value (the load started before the invalidate), but the cache stays in its post-invalidate state and the next lookup runs a fresh load. Aborting an in-flight load to make joiners retry is not possible with tokio::sync::OnceCell semantics.

Fetcher determinism

Concurrent callers may each pass a different fetcher closure, but only the first one to reach the cell is invoked; the rest are dropped. Callers should pass fetchers that are pure functions of the key, otherwise only the winning caller’s side effects fire.

pub fn len(&self) -> usize

Current number of entries (including any that are expired but not yet evicted by a read or cleanup_expired).

pub fn is_empty(&self) -> bool

true if the cache holds zero entries.

pub fn capacity(&self) -> NonZeroUsize

Configured capacity (LRU bound).

pub fn pending_loads_count(&self) -> usize

Number of in-flight loads currently waiting to resolve.

Useful for ops monitoring (“is this pod stuck on slow fetchers?”) and for tests that verify get_or_try_insert_with cleans up its in-flight cells correctly even on panic-unwind.