Struct CacheManager 

pub struct CacheManager { /* private fields */ }

Multi-tier cache orchestrator

This is the main interface for cache operations. It coordinates lookups and writes across L1 and L2 cache tiers.

Implementations

impl CacheManager

pub fn new() -> Self

Create a new cache manager with default L1 and no L2

pub async fn with_l2(l2_config: L2Config) -> Self

Create a new cache manager with L1 and L2

pub async fn lookup(&self, request: &CacheableRequest) -> CacheLookupResult

Lookup a request in the cache

Flow

1. Check L1 (in-memory)
2. If miss, check L2 (Redis)
3. If L2 hit, populate L1
4. Return result

Performance

• L1 hit: <1ms
• L2 hit: 1-2ms

pub async fn store(&self, request: &CacheableRequest, response: CachedResponse)

Store a response in the cache

Writes to both L1 and L2 asynchronously (non-blocking). This should be called after receiving a response from the LLM provider.

Performance

Non-blocking, returns immediately. Cache writes happen in background.

pub async fn store_with_ttl( &self, request: &CacheableRequest, response: CachedResponse, l2_ttl_seconds: u64, )

Store with custom L2 TTL

pub async fn invalidate(&self, request: &CacheableRequest)

Invalidate a cache entry across all tiers

pub async fn clear_all(&self)

Clear all cache entries (use with caution!)

pub async fn health_check(&self) -> CacheHealthStatus

Check health of cache tiers

pub fn metrics_snapshot(&self) -> MetricsSnapshot

Get metrics snapshot

pub fn l1_entry_count(&self) -> u64

Get L1 cache entry count

pub async fn l2_approximate_size(&self) -> Option<usize>

Get L2 cache approximate size

pub fn has_l2(&self) -> bool

Check if L2 is configured and available

pub fn metrics(&self) -> &CacheMetrics

Get shared metrics instance

Trait Implementations

impl Clone for CacheManager

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Default for CacheManager

fn default() -> Self

Returns the “default value” for a type.