pub const DEFAULT_CAPACITY: usize = 256;Expand description
Default cache capacity (kernels).
Memory-ceiling note: each entry holds an Arc<EmittedPtx> whose text
is the emitted PTX string. Typical kernels emit ~5-15 KB of PTX (a
vector-add lands around 2 KB; a small fused matmul around 12 KB), so at
256 entries the steady-state L1 footprint is on the order of ~2.5 MB
(~10 KB PTX × 256) plus the per-entry BLAKE3 hash (32 B) and the
LRU policy queue (one CacheKey per entry, 24 B). Hostile or
pathological blueprints could push individual entries into the
multi-MB range — a deliberately unrolled blueprint emitting 10 MB of
PTX would push the 256-slot cache to ~2.5 GB. Operators expecting
adversarial workloads should clamp this via
KernelCache::with_capacity (lower) and pair it with the on-disk
L2 cache so cold lookups still hit a persisted path. The cache does
NOT enforce a per-entry byte limit; the count cap is the only knob.