Struct DistributedCache 

pub struct DistributedCache { /* private fields */ }

Distributed cache implementation with local fallback.

This cache combines local in-memory storage with distributed storage, providing fast access with shared state across multiple instances.

Examples

use ultrafast_models_sdk::cache::{DistributedCache, Cache, CachedResponse};
use std::time::Duration;

let cache = DistributedCache::new(500); // 500 local entries

let response = CachedResponse::new(chat_response, Duration::from_secs(3600));
cache.set("key", response, Duration::from_secs(3600));

if let Some(cached) = cache.get("key") {
    println!("Cache hit from distributed cache");
}

Implementations

impl DistributedCache

pub fn new(local_capacity: usize) -> Self

Create a new distributed cache with the specified local capacity.

Arguments

• local_capacity - Maximum number of entries in the local cache

Examples

use ultrafast_models_sdk::cache::DistributedCache;

let cache = DistributedCache::new(500);

Trait Implementations

impl Cache for DistributedCache

fn get(&self, key: &str) -> Option<CachedResponse>

Retrieve a cached response by key.

fn set(&self, key: &str, response: CachedResponse, ttl: Duration)

Store a response in the cache.

fn invalidate(&self, key: &str)

Remove a specific entry from the cache.

fn clear(&self)

Clear all entries from the cache.

fn size(&self) -> usize

Get the current number of entries in the cache.