Struct LlmCache 

pub struct LlmCache { /* private fields */ }

In-memory LRU cache for LLM responses.

Implementations

impl LlmCache

pub fn new(max_entries: usize) -> Self

Create a new cache with maximum entry count.

pub fn cache_key(prompt: &str, system: Option<&str>, seed: Option<u64>) -> u64

Compute cache key from prompt + system + seed.

pub fn get(&self, key: u64) -> Option<String>

Get a cached response.

pub fn insert(&self, key: u64, content: String)

Insert a response into the cache.

pub fn len(&self) -> usize

Number of entries in cache.

pub fn is_empty(&self) -> bool

Whether the cache is empty.

pub fn clear(&self)

Clear all entries.