Struct EmbeddingCache 

pub struct EmbeddingCache { /* private fields */ }

Thread-safe LRU embedding cache with bytes-based capacity

Implementations

impl EmbeddingCache

pub fn new(max_bytes: usize) -> Self

Create a new cache with the specified byte capacity

Arguments

• max_bytes: Maximum bytes to use for embeddings
  • Default recommendation: 100MB (~25K embeddings @ 1536 dims)
  • Each 1536-dim embedding uses 6144 bytes (1536 * 4)

pub fn default_capacity() -> Self

Create a cache with default capacity (100MB)

pub fn get(&self, key: &str) -> Option<Arc<[f32]>>

Get an embedding from the cache

Returns Arc clone (cheap pointer copy, not vector copy)

pub fn put(&self, key: String, embedding: Vec<f32>)

Insert an embedding into the cache

If the key already exists, the embedding is updated and moved to front. If capacity is exceeded, least recently used entries are evicted.

pub fn stats(&self) -> EmbeddingCacheStats

Get cache statistics

pub fn clear(&self)

Clear all entries from the cache

pub fn len(&self) -> usize

Get the number of entries in the cache

pub fn is_empty(&self) -> bool

Check if the cache is empty

Trait Implementations

impl Default for EmbeddingCache

fn default() -> Self

Returns the “default value” for a type.