Struct InMemoryCache 

pub struct InMemoryCache { /* private fields */ }

Implementations

impl InMemoryCache

pub fn unbounded() -> Self

Unbounded cache (grows indefinitely)

Suitable for:

• Testing/development
• Short-lived processes
• Known input sets

pub fn with_max_entries(max_entries: u64) -> Self

Bounded by entry count

Evicts least-recently-used entries when full.

Example: 10,000 entries ≈ 60 MB (at 1536 dims/embedding)

pub fn with_max_memory_mb(mb: u64) -> Self

Bounded by approximate memory usage

Weights entries by vector.len() * 4 bytes (f32 size).

Example: 100 MB for ~17,000 embeddings (at 1536 dims)

Trait Implementations

impl Cache for InMemoryCache

fn get<'life0, 'life1, 'async_trait>( &'life0 self, input: &'life1 str, ) -> Pin<Box<dyn Future<Output = Result<Option<Vec<f32>>>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

fn put<'life0, 'life1, 'async_trait>( &'life0 self, input: &'life1 str, embedding: Vec<f32>, ) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

fn init<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

impl Default for InMemoryCache

fn default() -> Self

Default: bounded to 10,000 entries (~60 MB)