Skip to main content

Module response_cache

Module response_cache 

Source
Expand description

Response cache — avoid burning tokens on repeated prompts.

Stores LLM responses in a separate SQLite table keyed by a SHA-256 hash of (model, system_prompt_hash, user_prompt). Entries expire after a configurable TTL (default: 1 hour). The cache is optional and disabled by default — users opt in via [memory] response_cache_enabled = true.

Structs§

ResponseCache
Two-tier response cache: in-memory LRU (hot) + SQLite (warm).