Struct CachedEmbeddingService 

pub struct CachedEmbeddingService<S> { /* private fields */ }

Unstable: caching strategy and constructor API may change; foundation-internal use only.

LRU-caching wrapper around an embedding service.

It preserves input order while reusing embeddings with matching model configuration and role. See docs/service.md for the lookup and fill algorithm.

Implementations

impl<S: EmbeddingService> CachedEmbeddingService<S>

pub fn new(inner: Arc<S>, cache_capacity: usize) -> Self

Unstable: constructor signature may change when cache config becomes a struct.

Arguments

• inner - The underlying embedding service
• cache_capacity - Maximum number of embeddings to cache

pub fn with_default_cache(inner: Arc<S>) -> Self

Unstable: constructor signature may change when cache config becomes a struct.

pub fn cache_stats(&self) -> CacheStats

Unstable: returns internal CacheStats type which is itself Unstable.

pub fn clear_cache(&self)

Unstable: internal cache management; API subject to change.

Trait Implementations

impl<S: EmbeddingService + 'static> EmbeddingService for CachedEmbeddingService<S>

fn embed_with_role<'life0, 'life1, 'async_trait>( &'life0 self, texts: &'life1 [String], model: EmbeddingModel, role: EmbeddingRole, ) -> Pin<Box<dyn Future<Output = Result<Vec<Vec<f32>>>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Override: cache under the role key rather than prefixing here.

embed_query and embed_passage reach this through their trait defaults, so all three role paths share one cache-aware implementation.

fn embed<'life0, 'life1, 'async_trait>( &'life0 self, texts: &'life1 [String], model: EmbeddingModel, ) -> Pin<Box<dyn Future<Output = Result<Vec<Vec<f32>>>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Stable: generate embeddings for multiple texts.

fn supports_model(&self, model: EmbeddingModel) -> bool

Stable: check if the service supports a given model.

fn name(&self) -> &'static str

Stable: get the name/identifier of this service.

fn embed_one<'life0, 'life1, 'async_trait>( &'life0 self, text: &'life1 str, model: EmbeddingModel, ) -> Pin<Box<dyn Future<Output = Result<Vec<f32>>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Stable: generate an embedding for a single text.

fn embed_query<'life0, 'life1, 'async_trait>( &'life0 self, texts: &'life1 [String], model: EmbeddingModel, ) -> Pin<Box<dyn Future<Output = Result<Vec<Vec<f32>>>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Stable: embed query texts after applying the model’s query instruction.

fn embed_passage<'life0, 'life1, 'async_trait>( &'life0 self, texts: &'life1 [String], model: EmbeddingModel, ) -> Pin<Box<dyn Future<Output = Result<Vec<Vec<f32>>>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Stable: embed passages after applying the model’s document instruction.

fn model_config(&self, model: EmbeddingModel) -> ModelConfig

Unstable: returns the effective configuration used for this model’s cache keys.