Struct SemanticCache 

pub struct SemanticCache { /* private fields */ }

A cache that uses semantic similarity to find matches. Useful when prompts are slightly different but intent is the same.

Implementations

impl SemanticCache

pub fn new() -> Result<Self>

Create a new semantic cache with default embedding model.

pub fn with_threshold(self, threshold: f32) -> Self

Set similarity threshold (0.0 to 1.0).

Trait Implementations

impl Cache for SemanticCache

fn get(&self, prompt: &str) -> Option<String>

Try to retrieve a cached response for a prompt.

fn set(&self, prompt: &str, response: String)

Store a response in the cache.