Struct AsyncGlobalCache 

pub struct AsyncGlobalCache<'a, R: Clone> { /* private fields */ }

A thread-safe async global cache with configurable eviction policies and TTL support.

This cache is designed specifically for async/await contexts and uses lock-free concurrent data structures (DashMap) for optimal performance under high concurrency.

Type Parameters

• R - The type of values stored in the cache. Must implement Clone.

Features

• Lock-free reads/writes: Uses DashMap for concurrent access without blocking
• Eviction policies: FIFO, LRU (default), LFU, ARC, Random, and TLRU
  • FIFO: First In, First Out - simple and predictable
  • LRU: Least Recently Used - evicts least recently accessed entries
  • LFU: Least Frequently Used - evicts least frequently accessed entries
  • ARC: Adaptive Replacement Cache - hybrid policy combining recency and frequency
  • Random: Random replacement - O(1) eviction with minimal overhead
  • TLRU: Time-aware LRU - combines recency, frequency, and age factors
    • Customizable with frequency_weight parameter
    • Formula: score = frequency^weight × position × age_factor
    • frequency_weight < 1.0: Emphasize recency (time-sensitive data)
    • frequency_weight > 1.0: Emphasize frequency (popular content)
• Cache limits: Entry count limits (limit) and memory-based limits (max_memory)
• TTL support: Automatic expiration of entries based on age
• Statistics: Optional cache hit/miss tracking (with stats feature)
• Frequency tracking: For LFU, ARC, and TLRU policies
• Memory estimation: Support for memory-based eviction (requires MemoryEstimator)

Cache Entry Structure

Each cache entry is stored as a tuple: (value, timestamp, frequency)

• value: The cached value of type R
• timestamp: Unix timestamp when the entry was created (for TTL and TLRU age factor)
• frequency: Access counter for LFU, ARC, and TLRU policies

Eviction Behavior

When the cache reaches its limit (entry count or memory), entries are evicted according to the configured policy:

• FIFO: Oldest entry (first in order queue) is evicted
• LRU: Least recently accessed entry (first in order queue) is evicted
• LFU: Entry with lowest frequency counter is evicted
• ARC: Entry with lowest score (frequency × position_weight) is evicted
• Random: Randomly selected entry is evicted
• TLRU: Entry with lowest score (frequency^weight × position × age_factor) is evicted

Performance Characteristics

• Get: O(1) for cache lookup, O(n) for LRU/ARC/TLRU reordering
• Insert: O(1) for FIFO/Random, O(n) for LRU/LFU/ARC/TLRU eviction
• Memory: O(n) where n is the number of cached entries

Thread Safety

This structure is fully thread-safe and can be shared across multiple async tasks. The underlying DashMap provides lock-free concurrent access, while the order queue uses a Mutex for coordination.

Examples

Basic Usage

use cachelito_core::{AsyncGlobalCache, EvictionPolicy};
use dashmap::DashMap;
use parking_lot::Mutex;
use std::collections::VecDeque;

let cache = DashMap::new();
let order = Mutex::new(VecDeque::new());
let async_cache = AsyncGlobalCache::new(
    &cache,
    &order,
    Some(100),    // Max 100 entries
    None,         // No memory limit
    EvictionPolicy::LRU,
    Some(60),     // 60 second TTL
    None,         // Default frequency_weight for TLRU
);

// In async context:
if let Some(value) = async_cache.get("key") {
    println!("Cache hit: {}", value);
}

TLRU with Custom Frequency Weight

use cachelito_core::{AsyncGlobalCache, EvictionPolicy};

// Emphasize frequency over recency (good for popular content)
let async_cache = AsyncGlobalCache::new(
    &cache,
    &order,
    Some(100),
    None,
    EvictionPolicy::TLRU,
    Some(300),
    Some(1.5),    // frequency_weight > 1.0
);

// Emphasize recency over frequency (good for time-sensitive data)
let async_cache = AsyncGlobalCache::new(
    &cache,
    &order,
    Some(100),
    None,
    EvictionPolicy::TLRU,
    Some(300),
    Some(0.3),    // frequency_weight < 1.0
);

With Memory Limits

use cachelito_core::{AsyncGlobalCache, EvictionPolicy, MemoryEstimator};

let async_cache = AsyncGlobalCache::new(
    &cache,
    &order,
    Some(1000),
    Some(100 * 1024 * 1024), // 100MB max
    EvictionPolicy::LRU,
    Some(300),
    None,
);

// Insert with memory tracking (requires MemoryEstimator implementation)
async_cache.insert_with_memory("key", value);

Implementations

impl<'a, R: Clone> AsyncGlobalCache<'a, R>

pub fn new( cache: &'a DashMap<String, (R, u64, u64)>, order: &'a Mutex<VecDeque<String>>, limit: Option<usize>, max_memory: Option<usize>, policy: EvictionPolicy, ttl: Option<u64>, frequency_weight: Option<f64>, window_ratio: Option<f64>, stats: &'a CacheStats, ) -> Self

Creates a new AsyncGlobalCache with statistics support.

This version is available when the stats feature is enabled.

pub fn get(&self, key: &str) -> Option<R>

Attempts to retrieve a value from the cache.

This method checks if the key exists, validates TTL expiration, updates access patterns based on the eviction policy, and records statistics.

Arguments

• key - The cache key to look up

Returns

• Some(R) - The cached value if found and not expired
• None - If the key doesn’t exist or has expired

Behavior by Policy

• FIFO: No updates on cache hit (order remains unchanged)
• LRU: Moves the key to the end of the order queue (most recently used)
• LFU: Increments the frequency counter for the entry
• ARC: Increments frequency counter and updates position in order queue
• Random: No updates on cache hit
• TLRU: Increments frequency counter and updates position in order queue

TTL Expiration

If a TTL is configured and the entry has expired:

• The entry is removed from both the cache and order queue
• A cache miss is recorded (if stats feature is enabled)
• None is returned

Statistics

When the stats feature is enabled:

• Cache hits are recorded when a valid entry is found
• Cache misses are recorded when the key doesn’t exist or has expired

Examples

// Check for cached user
if let Some(user) = async_cache.get("user:123") {
    println!("Found user: {:?}", user);
} else {
    println!("Cache miss - need to fetch from database");
}

Performance

• FIFO, Random: O(1) - no reordering needed
• LRU, ARC, TLRU: O(n) - requires finding and moving key in order queue
• LFU: O(1) - only increments counter

pub fn insert(&self, key: &str, value: R)

Inserts a value into the cache.

This method handles cache limit enforcement and eviction according to the configured policy. If the cache is full, it evicts an entry before inserting the new one.

Arguments

• key - The cache key
• value - The value to cache

Eviction Behavior

• FIFO: Evicts the oldest inserted entry (front of queue)
• LRU: Evicts the least recently used entry (front of queue)
• LFU: Evicts the entry with the lowest frequency counter
• ARC: Evicts based on a hybrid score of frequency and recency

Thread Safety

This method uses locks to ensure consistency between the cache and the order queue. The order lock is held during eviction and insertion to prevent race conditions.

Note

This method does NOT require MemoryEstimator trait. It only handles entry-count limits. If max_memory is configured, use insert_with_memory() instead, which requires the type to implement MemoryEstimator.

Examples

// Insert a new value
async_cache.insert("user:123", user_data);

// Update existing value
async_cache.insert("user:123", updated_user_data);

pub fn stats(&self) -> &CacheStats

Returns a reference to the cache statistics.

This method is only available when the stats feature is enabled.

Available Metrics

The returned CacheStats provides:

• hits(): Number of successful cache lookups
• misses(): Number of cache misses (key not found or expired)
• hit_rate(): Ratio of hits to total accesses (0.0 to 1.0)
• total_accesses(): Total number of get operations

Thread Safety

Statistics use atomic counters (AtomicU64) and can be safely accessed from multiple async tasks without additional synchronization.

Examples

// Get basic statistics
let stats = async_cache.stats();
println!("Hits: {}", stats.hits());
println!("Misses: {}", stats.misses());
println!("Hit rate: {:.2}%", stats.hit_rate() * 100.0);

// Monitor cache performance
let total = stats.total_accesses();
if total > 1000 && stats.hit_rate() < 0.5 {
    println!("Warning: Low cache hit rate");
}

See Also

• CacheStats - The statistics structure
• crate::stats_registry::get() - Access stats by cache name

impl<'a, R: Clone + MemoryEstimator> AsyncGlobalCache<'a, R>

pub fn insert_with_memory(&self, key: &str, value: R)

Insert with memory limit support.

This method requires R to implement MemoryEstimator and handles both memory-based and entry-count-based eviction.

Use this method when max_memory is configured in the cache.

Arguments

• key - The cache key
• value - The value to cache (must implement MemoryEstimator)

Memory Management

The method calculates the memory footprint of all cached entries and evicts entries as needed to stay within the max_memory limit. Eviction follows the configured policy.

Safety Check

If the value to be inserted is larger than max_memory, the insertion is skipped entirely to avoid infinite eviction loops. This ensures the cache respects the memory limit even if individual values are very large.

Eviction Behavior by Policy

When memory limit is exceeded:

• FIFO/LRU: Evicts from front of order queue
• LFU: Evicts entry with lowest frequency
• ARC: Evicts based on hybrid score (frequency × position_weight)
• TLRU: Evicts based on TLRU score (frequency^weight × position × age_factor)
• Random: Evicts randomly selected entry

The eviction loop continues until there’s enough memory for the new value.

Entry Count Limit

After satisfying memory constraints, this method also checks the entry count limit (if configured) and evicts additional entries if needed.

Examples

use cachelito_core::MemoryEstimator;

// Type must implement MemoryEstimator
impl MemoryEstimator for MyLargeStruct {
    fn estimate_memory(&self) -> usize {
        std::mem::size_of::<Self>() + self.data.capacity()
    }
}

// Insert with automatic memory-based eviction
async_cache.insert_with_memory("large_data", expensive_value);

Performance

• Memory calculation: O(n) - iterates all entries to sum memory
• Eviction: Varies by policy (see individual policy documentation)
• May evict multiple entries in one call if memory limit is tight