Struct CachedEmbeddings 

pub struct CachedEmbeddings { /* private fields */ }

A caching wrapper for embedding providers.

This wrapper adds a caching layer on top of any embedding provider to avoid regenerating embeddings for the same text. This is especially useful when:

• The same queries or texts are embedded multiple times
• Embeddings are expensive to compute (API calls, local models)
• You want to reduce API costs and latency

Cache Strategy

• Uses a simple in-memory HashMap with RwLock for thread-safety
• Cache key is the exact text string
• No cache eviction (unbounded cache - use with caution for large datasets)
• Thread-safe for concurrent access

Example

use ceylon_next::memory::vector::{EmbeddingProvider, CachedEmbeddings};
use std::sync::Arc;

#[tokio::main]
async fn main() {
    // Wrap any embedding provider with caching
    // let base_provider = OpenAIEmbeddings::new("api-key");
    // let cached = CachedEmbeddings::new(Arc::new(base_provider));

    // First call - computes embedding
    // let embedding1 = cached.embed("hello world").await.unwrap();

    // Second call - returns cached result
    // let embedding2 = cached.embed("hello world").await.unwrap();
}

Implementations

impl CachedEmbeddings

pub fn new(provider: Arc<dyn EmbeddingProvider>) -> Self

Creates a new cached embedding provider.

Arguments

• provider - The underlying embedding provider to wrap

pub fn with_capacity( provider: Arc<dyn EmbeddingProvider>, capacity: usize, ) -> Self

Creates a new cached embedding provider with pre-allocated cache capacity.

Arguments

• provider - The underlying embedding provider to wrap
• capacity - Initial capacity for the cache

pub async fn cache_size(&self) -> usize

Returns the number of cached embeddings.

pub async fn clear_cache(&self)

Clears all cached embeddings.

pub async fn is_cached(&self, text: &str) -> bool

Checks if a text is in the cache.

pub async fn preload(&self, texts: &[String]) -> Result<(), String>

Preloads embeddings into the cache.

This can be useful for warming up the cache with known queries.

Arguments

• texts - Texts to preload into cache

Trait Implementations

impl EmbeddingProvider for CachedEmbeddings

fn embed<'life0, 'life1, 'async_trait>( &'life0 self, text: &'life1 str, ) -> Pin<Box<dyn Future<Output = Result<Vec<f32>, String>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Generates an embedding vector for the given text.

fn embed_batch<'life0, 'life1, 'async_trait>( &'life0 self, texts: &'life1 [String], ) -> Pin<Box<dyn Future<Output = Result<Vec<Vec<f32>>, String>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Generates embeddings for multiple texts in batch.

fn dimension(&self) -> usize

Returns the dimensionality of the embeddings produced by this provider.

fn model_name(&self) -> &str

Returns the model name or identifier.