Struct SemanticCache 

pub struct SemanticCache { /* private fields */ }

Semantic cache using embeddings for similarity matching

Implementations

impl SemanticCache

pub fn new( embedding_provider: Box<dyn EmbeddingProvider>, threshold: SimilarityThreshold, max_size: usize, ) -> Self

Create a new semantic cache

Arguments

• embedding_provider - Provider for generating embeddings
• threshold - Minimum similarity score for cache hit (0.0 to 1.0)
• max_size - Maximum number of entries to cache

pub async fn stats(&self) -> SemanticCacheStats

Get cache statistics

pub async fn clear(&self)

Clear the cache and reset statistics

pub async fn get(&self, prompt: &str) -> Option<LlmResponse>

Try to get a cached response for a prompt

pub async fn put(&self, prompt: String, response: LlmResponse)

Store a response in the cache