Struct CachedEmbeddingService 

pub struct CachedEmbeddingService<S> { /* private fields */ }

Unstable: caching strategy and constructor API may change; foundation-internal use only.

Caching wrapper around an embedding service.

Wraps any EmbeddingService implementation with LRU caching. Identical texts (with the same model) will return cached embeddings instead of recomputing.

Example

use lattice_embed::{
    CachedEmbeddingService, NativeEmbeddingService, EmbeddingService,
    EmbeddingModel, EmbeddingCache,
};
use std::sync::Arc;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let inner = Arc::new(NativeEmbeddingService::new());
    let cached = CachedEmbeddingService::new(inner, 1000);

    // First call - computes and caches
    let emb1 = cached.embed_one("Hello", EmbeddingModel::default()).await?;

    // Second call - returns from cache
    let emb2 = cached.embed_one("Hello", EmbeddingModel::default()).await?;

    assert_eq!(emb1, emb2);
    Ok(())
}

Implementations

impl<S: EmbeddingService> CachedEmbeddingService<S>

pub fn new(inner: Arc<S>, cache_capacity: usize) -> Self

Unstable: constructor signature may change when cache config becomes a struct.

Arguments

• inner - The underlying embedding service
• cache_capacity - Maximum number of embeddings to cache

pub fn with_default_cache(inner: Arc<S>) -> Self

Unstable: constructor signature may change when cache config becomes a struct.

pub fn cache_stats(&self) -> CacheStats

Unstable: returns internal CacheStats type which is itself Unstable.

pub fn clear_cache(&self)

Unstable: internal cache management; API subject to change.

Trait Implementations

impl<S: EmbeddingService + 'static> EmbeddingService for CachedEmbeddingService<S>

fn embed<'life0, 'life1, 'async_trait>( &'life0 self, texts: &'life1 [String], model: EmbeddingModel, ) -> Pin<Box<dyn Future<Output = Result<Vec<Vec<f32>>>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Stable: generate embeddings for multiple texts.

fn supports_model(&self, model: EmbeddingModel) -> bool

Stable: check if the service supports a given model.

fn name(&self) -> &'static str

Stable: get the name/identifier of this service.

fn embed_one<'life0, 'life1, 'async_trait>( &'life0 self, text: &'life1 str, model: EmbeddingModel, ) -> Pin<Box<dyn Future<Output = Result<Vec<f32>>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Stable: generate an embedding for a single text.

fn model_config(&self, model: EmbeddingModel) -> ModelConfig

Unstable: returns the effective ModelConfig for a given model on this service.