Struct InMemoryCache 

pub struct InMemoryCache { /* private fields */ }

In-memory LLM response cache with optional TTL expiration.

Implementations

impl InMemoryCache

pub fn new() -> Self

Create a new cache with no TTL (entries never expire).

pub fn with_ttl(duration: Duration) -> Self

Create a new cache where entries expire after the given duration.

Trait Implementations

impl Default for InMemoryCache

fn default() -> Self

Returns the “default value” for a type.

impl LlmCache for InMemoryCache

fn get<'life0, 'life1, 'async_trait>( &'life0 self, key: &'life1 str, ) -> Pin<Box<dyn Future<Output = Result<Option<ChatResponse>, SynapticError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Look up a cached response by cache key.

fn put<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, key: &'life1 str, response: &'life2 ChatResponse, ) -> Pin<Box<dyn Future<Output = Result<(), SynapticError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait,

Store a response in the cache.

fn clear<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Result<(), SynapticError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Clear all entries from the cache.