Struct CacheManager 

pub struct CacheManager { /* private fields */ }

Cache manager for handling both Redis and in-memory caching.

Provides a unified interface for caching operations with automatic fallback from Redis to in-memory storage on failures.

Thread Safety

All operations are thread-safe and can be used concurrently.

Example

let config = CacheConfig {
    enabled: true,
    backend: CacheBackend::Redis { url: "redis://localhost:6379".to_string() },
    ttl: Duration::from_secs(3600),
    max_size: 1000,
};

let cache_manager = CacheManager::new(config).await?;

Implementations

impl CacheManager

pub async fn new(config: CacheConfig) -> Result<Self, GatewayError>

Create a new cache manager with the specified configuration.

Initializes the cache manager with either Redis or in-memory backend based on the configuration. Falls back to in-memory if Redis is unavailable.

Arguments

• config - Cache configuration including backend and settings

Returns

Returns a new CacheManager instance.

Errors

Returns an error if the cache manager cannot be initialized.

pub async fn get(&self, key: &str) -> Option<Value>

Retrieve a value from the cache.

Attempts to retrieve a value from Redis first, then falls back to in-memory storage. Returns None if the key doesn’t exist or has expired.

Arguments

• key - The cache key to retrieve

Returns

Returns the cached value if found and not expired, None otherwise.

pub async fn set(&self, key: &str, value: Value, custom_ttl: Option<Duration>)

Cache a value in the cache.

Attempts to cache a value using Redis first, then falls back to in-memory storage.

Arguments

• key - The cache key to set
• value - The value to cache as a JSON value
• custom_ttl - Optional custom TTL for this entry, overrides config

Example

let key = CacheKeyBuilder::chat_completion_key("gpt-4", &messages_hash);
cache_manager.set(&key, response_data, None).await;

pub async fn invalidate(&self, key: &str)

Invalidate a single cache entry.

Attempts to invalidate a cache entry using Redis first, then falls back to in-memory storage.

Arguments

• key - The cache key to invalidate

Example

let key = CacheKeyBuilder::chat_completion_key("gpt-4", &messages_hash);
cache_manager.invalidate(&key).await;

pub async fn clear(&self)

Clear the entire cache.

Attempts to clear the cache using Redis first, then falls back to in-memory storage.

Example

cache_manager.clear().await;

pub async fn stats(&self) -> CacheStats

Get current cache statistics.

Returns statistics about the cache, including total entries, expired entries, and memory usage.

Returns

Returns a CacheStats struct containing the statistics.

pub fn has_redis(&self) -> bool

Returns true if Redis backend is available

pub async fn incr_with_expiry( &self, key: &str, expiry_secs: usize, ) -> Result<i64, GatewayError>

Atomically increment a key with an expiry. If the key is new, expiry is set.

pub async fn incr_by_with_expiry( &self, key: &str, by: i64, expiry_secs: usize, ) -> Result<i64, GatewayError>

Atomically increment a key by a specific amount with an expiry. If the key is new, expiry is set.

Trait Implementations

impl Debug for CacheManager

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.