Struct MetadataCachePolicy 

pub struct MetadataCachePolicy { /* private fields */ }

Every caching decision, in one place. Given an info’s size and the current CacheMode, the policy decides whether that info is cached at all (should_cache) and, through its capacity, when the cache must evict to stay bounded. How those decisions are carried out (lookups, eviction) is the cache’s job and differs per runtime; what to do lives here.

max_entries and max_cached_size are the backend-tunable knobs.

Implementations

impl MetadataCachePolicy

pub fn new(max_entries: usize, max_cached_size: usize) -> Self

Build a policy in CacheMode::Normal. max_entries caps the number of cached entries; max_cached_size (bytes) is the largest info still worth caching in normal operation.

pub fn mode(&mut self, mode: CacheMode)

Switch the mode driving the policy’s decisions (see CacheMode).

pub fn should_cache(&self, size: usize) -> bool

Whether an info of size bytes should go through the cache at all. In CacheMode::Normal a too-large info (or a disabled cache) is skipped: the runtime should create it directly, without a lookup or a store. In CacheMode::Capture everything is cached so the capture window stays warm.

pub fn capacity(&self) -> Option<usize>

The entry bound the cache must hold to, or None when unbounded. In CacheMode::Capture the cache is unbounded and never evicts (dropping an entry could free a buffer a recorded launch still needs); in CacheMode::Normal it is capped at max_entries.

pub fn pins_entries(&self) -> bool

Whether entries touched right now must be pinned to the graph being captured. True in CacheMode::Capture: a graph records the device pointer of every info buffer its launches touch, so those entries must not be evicted for the graph’s lifetime — even after the cache returns to CacheMode::Normal, and even if the buffer lives in the regular (non-persistent) pool and so is not otherwise retained by the graph.

Trait Implementations

impl Clone for MetadataCachePolicy

fn clone(&self) -> MetadataCachePolicy

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for MetadataCachePolicy

impl Debug for MetadataCachePolicy

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for MetadataCachePolicy

fn default() -> Self

Returns the “default value” for a type.