Module cache 

Compiled-kernel cache.

The cache is keyed by (blueprint_fingerprint, sm_version). LRU eviction caps the entry count. On a cache hit the PTX text is reused without re-emitting (cheap) and without re-compiling via ptxas/cust (expensive — 10-50 ms for non-trivial kernels).

Storage: a dashmap::DashMap holds the actual (CacheKey, CachedKernel) entries — get / put / len go straight through the lock-free per-shard locks so the hot path is uncontended under multi-threaded dispatch. A separate parking_lot::Mutex<LruCache<CacheKey, ()>> is the eviction queue — touched on put (insert/promote) and on get (promote) and used to compute which key to evict when the soft cap is exceeded. Splitting storage from policy means lookups never block on the eviction mutex, only inserts that need to evict do. The get read path is lock-free on contention; LRU promotion is best-effort under load — contended promotions are skipped, so eviction order is approximately (not strictly) LRU when many readers race.

Mutex poisoning recovery: every lock acquisition uses into_inner on a poisoned guard (after emitting a tracing::error!) rather than the prior .expect("cache poisoned") panic — a single panic on any thread used to poison the entire cache for the rest of the process.

Structs

CacheKey

Cache key.

CachedKernel

Cached PTX module entry.

CompiledHandle

Compiled-module handle. On CUDA hosts this would hold cust::module::Module; for the no-CUDA path it is just ().

DiskCacheConfig

Configuration for the on-disk L2 cache.

KernelCache

Thread-safe LRU cache of compiled kernels backed by dashmap::DashMap.

KernelCacheConfig

Construction-time configuration for KernelCache.

KeyFingerprint

16-hex-char fingerprint of an HMAC key, as it appears in on-disk filenames.

Constants

DEFAULT_CAPACITY

Default cache capacity (kernels).