Struct ByteLruCache 

pub struct ByteLruCache<K: Eq + Hash + Clone, V> { /* private fields */ }

Byte-limited LRU cache with an optional entry cap.

Unlike an entry-count-limited LRU, this cache tracks the resident byte cost of each value (via ResidentBytes) and evicts the least-recently-used entries until the total resident bytes fit under max_bytes. This is the correct policy for large-scale payloads where a single cache entry (e.g. an n*K distance matrix) can itself be multiple gigabytes and an entry-count cap would silently blow the memory budget. Small entry caps are still useful for payloads with known shape, such as owned PC data matrices shared across model blocks.

Implementations

impl<K: Eq + Hash + Clone, V: Clone + ResidentBytes> ByteLruCache<K, V>

pub fn new(max_bytes: usize) -> Self

pub fn with_max_entries(max_bytes: usize, max_entries: usize) -> Self

pub fn new_sharded(max_bytes: usize, shard_count: usize) -> Self

Like new but partitions the cache across shard_count independently-locked LRU shards to cut lock contention under heavy concurrent access. The byte budget is divided evenly across shards, so this is only appropriate for caches holding many small entries.

pub fn with_max_entries_sharded( max_bytes: usize, max_entries: usize, shard_count: usize, ) -> Self

Like with_max_entries but sharded; see new_sharded.

pub fn get(&self, key: &K) -> Option<V>

pub fn insert(&self, key: K, value: V)

pub fn resident_bytes(&self) -> usize

pub const fn max_bytes(&self) -> usize

pub fn len(&self) -> usize

pub fn is_empty(&self) -> bool

pub fn clear(&self)

Trait Implementations

impl<K: Eq + Hash + Clone, V: Clone + ResidentBytes> Debug for ByteLruCache<K, V>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.