Struct ResponseCache 

pub struct ResponseCache { /* private fields */ }

Implementations

impl ResponseCache

pub fn new(pool: SqlitePool, ttl_secs: u64) -> Self

pub async fn get(&self, key: &str) -> Result<Option<String>, MemoryError>

Look up a cached response by key. Returns None if not found or expired.

Errors

Returns an error if the database query fails.

pub async fn put( &self, key: &str, response: &str, model: &str, ) -> Result<(), MemoryError>

Store a response in the cache with TTL.

Errors

Returns an error if the database insert fails.

pub async fn get_semantic( &self, embedding: &[f32], embedding_model: &str, similarity_threshold: f32, max_candidates: u32, ) -> Result<Option<(String, f32)>, MemoryError>

Semantic similarity-based cache lookup.

Fetches up to max_candidates non-expired rows with matching embedding_model, deserializes each embedding, computes cosine similarity against the query vector, and returns the response with the highest score if it meets similarity_threshold.

Returns (response_text, score) on hit, None on miss.

Errors

Returns an error if the database query fails.

pub async fn put_with_embedding( &self, key: &str, response: &str, model: &str, embedding: &[f32], embedding_model: &str, ) -> Result<(), MemoryError>

Store a response with an embedding vector for future semantic matching.

Uses INSERT OR REPLACE — updates the embedding on existing rows.

Errors

Returns an error if the database insert fails.

pub async fn invalidate_embeddings_for_model( &self, old_model: &str, ) -> Result<u64, MemoryError>

Set embedding = NULL for all rows with the given embedding_model.

Called when the embedding model changes to prevent cross-model false hits. Returns the number of rows updated.

Errors

Returns an error if the database update fails.

pub async fn cleanup( &self, current_embedding_model: &str, ) -> Result<u64, MemoryError>

Two-phase cleanup: delete expired rows, then NULL-ify stale embeddings.

Phase 1: DELETE rows where expires_at <= now. Phase 2: UPDATE rows where embedding_model != current_embedding_model to NULL out the embedding columns. Exact-match data (cache_key, response) is preserved.

Returns the total number of rows affected (deleted + updated).

Errors

Returns an error if either database operation fails.

pub async fn cleanup_expired(&self) -> Result<u64, MemoryError>

Delete expired cache entries. Returns the number of rows deleted.

Errors

Returns an error if the database delete fails.

pub fn compute_key(last_user_message: &str, model: &str) -> String

Compute a deterministic cache key from the last user message and model name using blake3.

The key intentionally ignores conversation history so that identical user messages produce cache hits regardless of what preceded them. This is the desired behavior for a short-TTL response cache, but it means context-dependent questions (e.g. “Explain this”) may return a cached response from a different context. The TTL bounds staleness.