Struct CachedEmbeddingProvider 

pub struct CachedEmbeddingProvider { /* private fields */ }

Wraps an EmbeddingProvider with a file-backed cache. Embeddings are keyed by SHA-256 of the input text.

Implementations

impl CachedEmbeddingProvider

pub fn new(inner: Box<dyn EmbeddingProvider>, path: impl Into<PathBuf>) -> Self

pub fn flush(&self)

Flush cache to disk if dirty.

impl CachedEmbeddingProvider

pub fn embed_cached(&mut self, text: &str) -> Result<Embedding, EmbeddingError>

Embed with caching — stores result in cache.

pub fn embed_batch_cached_paced( &mut self, texts: &[&str], chunk_size: usize, inter_batch_delay: Duration, ) -> Result<Vec<Embedding>, EmbeddingError>

Embed many texts at once, calling inner.embed_batch on cache misses. Cache hits are served from memory. Misses are sent in chunks of chunk_size to keep individual requests under typical provider payload limits.

Two robustness properties matter when a remote provider is rate-limited:

• Progressive caching. Each successful batch is committed to the in-memory cache and flushed to disk immediately. If the next batch trips a 429, the function still returns Err — but the partial work done so far is durable, so the next process restart picks up exactly where the crash left off.
• Inter-batch pacing. Between batch calls we sleep inter_batch_delay. With Mistral’s free-tier 1 req/s ceiling, a ~1100 ms sleep keeps us comfortably under the limit; paid tiers can pass Duration::ZERO to skip pacing.

Order of results matches order of texts.

pub fn embed_batch_cached( &mut self, texts: &[&str], chunk_size: usize, ) -> Result<Vec<Embedding>, EmbeddingError>

Backward-compatible wrapper: no inter-batch sleep, single final flush. Prefer embed_batch_cached_paced for rate-limited providers.

Trait Implementations

impl Drop for CachedEmbeddingProvider

fn drop(&mut self)

Executes the destructor for this type.

impl EmbeddingProvider for CachedEmbeddingProvider

fn dimensions(&self) -> usize

fn embed(&self, text: &str) -> Result<Embedding, EmbeddingError>

fn embed_batch(&self, texts: &[&str]) -> Result<Vec<Embedding>, EmbeddingError>