Skip to main content

Crate byte_lru_cache

Crate byte_lru_cache 

Source
Expand description

§byte-lru-cache

Crates.io Documentation CI License Downloads

Thread-safe LRU caching governed by an explicit byte budget.

byte-lru-cache stores typed values behind Arc and evicts the least-recently-used entries when the sum of caller-supplied weights exceeds the configured ceiling. It performs no I/O, starts no tasks, and has no runtime dependency.

§Why a byte budget?

An entry-count limit is a poor memory bound when values have different sizes. This crate makes the accounting decision explicit at insertion time:

use std::num::NonZeroU64;
use std::sync::Arc;

use byte_lru_cache::ByteLruCache;

let cache = ByteLruCache::new(1024, "parsed-values");
cache.insert(
    "answer",
    Arc::new(42_u32),
    NonZeroU64::new(4).expect("weight is positive"),
);

assert_eq!(cache.get(&"answer").as_deref(), Some(&42));
assert_eq!(cache.current_bytes(), 4);

Weights must be positive. This prevents zero-weight entries from bypassing the memory bound. A value whose weight is larger than the current ceiling is inserted and then immediately evicted.

The cache tracks the supplied weights, not the allocator’s complete heap footprint. Callers should provide conservative weights for their values.

§Runtime resizing

The ceiling can be lowered or raised while the cache is in use. Lowering it evicts entries synchronously before the method returns:

use std::sync::Arc;

use byte_lru_cache::ByteLruCache;

let cache = ByteLruCache::new(1024, "parsed-values");
cache.insert(
    "answer",
    Arc::new(42_u32),
    std::num::NonZeroU64::new(4).expect("weight is positive"),
);

cache.set_max_bytes(512);
assert!(cache.current_bytes() <= 512);

The cache is independent of any process-wide memory governor by default. A governor can sample current_bytes() and call set_max_bytes() without adding a runtime, I/O, or policy dependency to this crate — and with the optional memory-budget feature, ByteLruCache implements memory_budget::Resizable directly, so registering the cache with the memory-budget coordinator needs no adapter:

byte-lru-cache = { version = "0.1", features = ["memory-budget"] }

§Concurrency

ByteLruCache<K, V> is safe to share through Arc when K and V are Send + Sync. Cache operations serialize only the short LRU bookkeeping critical section. Returned values remain independently owned through their Arc handles.

§License

Licensed under either of:

A thread-safe LRU cache whose admission is governed by a byte budget.

Structs§

ByteLruCache
A thread-safe LRU cache governed by a caller-supplied byte budget.
CacheStats
Monotonic counters collected by a ByteLruCache.