Struct EmbeddingCache 

pub struct EmbeddingCache { /* private fields */ }

Unstable: internal LRU caching mechanism; shard count and eviction policy may change.

Embedding cache with sharded LRU eviction policy.

Thread-safe cache for storing computed embeddings. Uses Blake3 hashing for fast, collision-resistant cache keys. Internally sharded into 16 independent LRU caches to reduce write-lock contention.

Disabling

Pass capacity=0 to disable caching. All cache operations become no-ops (no locking, no hashing work beyond key construction).

Example

use lattice_embed::{EmbeddingCache, EmbeddingModel, ModelConfig};

let cache = EmbeddingCache::new(1000);

// Cache miss - no embedding stored yet
let key = cache.compute_key("Hello, world!", ModelConfig::new(EmbeddingModel::BgeSmallEnV15));
assert!(cache.get(&key).is_none());

// Store embedding
let embedding = vec![0.1, 0.2, 0.3];
cache.put(key, embedding.clone());

// Cache hit — returns Arc<[f32]>
let cached = cache.get(&key).unwrap();
assert_eq!(&*cached, &embedding[..]);

Implementations

impl EmbeddingCache

pub fn new(capacity: usize) -> Self

Unstable: constructor signature may change when shard count becomes configurable.

The capacity is divided equally across 16 internal shards. Each shard independently manages its own LRU eviction.

Arguments

• capacity - Maximum total number of embeddings to cache. Use 0 to disable caching.

pub fn with_default_capacity() -> Self

Unstable: convenience constructor; subject to change with cache redesign.

pub fn compute_key(&self, text: &str, model_config: ModelConfig) -> [u8; 32]

Unstable: key scheme (Blake3 + EmbeddingKey canonical bytes) may change; don’t store keys across sessions.

Uses Blake3 hashing for fast, collision-resistant keys. The key includes the model name, revision, and active dimension from the ModelConfig, so different MRL truncations produce different cache keys.

pub fn get(&self, key: &[u8; 32]) -> Option<Arc<[f32]>>

Unstable: return type (Arc<[f32]>) may change to a newtype; internal cache API.

Returns Some(Arc<[f32]>) if found (cheap refcount bump), None otherwise. Updates per-shard hit/miss counters for metrics.

pub fn put(&self, key: [u8; 32], embedding: Vec<f32>)

Unstable: internal cache storage method; interface may change.

Converts the Vec into Arc<[f32]> for shared-ownership storage. If the shard is at capacity, its least recently used entry is evicted.

pub fn get_many(&self, keys: &[[u8; 32]]) -> Vec<Option<Arc<[f32]>>>

Unstable: batch cache access; return type may change with cache redesign.

Returns a vector of Option<Arc<[f32]>> for each key, in the same order. Each hit is an O(1) refcount bump (no data copy).

pub fn put_many(&self, entries: Vec<([u8; 32], Vec<f32>)>)

Unstable: batch cache storage; interface may change with cache redesign.

Converts each Vec into Arc<[f32]> for shared-ownership storage.

pub fn stats(&self) -> CacheStats

Unstable: returns CacheStats which is itself Unstable; metrics shape may evolve.

Aggregates per-shard counters. The size field is the sum of all shard sizes.

pub fn per_shard_stats(&self) -> Vec<ShardStats>

Unstable: internal monitoring hook; shard count and ShardStats shape may change.

Returns a vector of (size, hits, misses) tuples, one per shard.

pub fn clear(&self)

Unstable: internal cache management; may be removed in favor of capacity-based eviction.

pub fn is_enabled(&self) -> bool

Unstable: internal state query; may be removed when zero-capacity is the only disable path.

Trait Implementations

impl Default for EmbeddingCache

fn default() -> Self

Returns the “default value” for a type.