Module cached_backend 

Tiered caching layer for optimal performance with distributed state

This module implements a three-tiered caching architecture:

• L1: In-memory cache (moka) - fastest, local to instance
• L2: Redis cache - shared across instances, faster than DB
• L3: PostgreSQL/Sled backend - durable storage, source of truth

Architecture

┌─────────────────────────────────────────────┐
│         CachedStateBackend<B>              │
│                                             │
│  ┌──────────────────────────────────────┐  │
│  │  L1: In-Memory Cache (moka)          │  │
│  │  - Thread-safe, lock-free            │  │
│  │  - LRU eviction                      │  │
│  │  - TTL support                       │  │
│  │  - ~1-10μs latency                   │  │
│  └──────────────────────────────────────┘  │
│              ↓ miss                         │
│  ┌──────────────────────────────────────┐  │
│  │  L2: Redis Cache                     │  │
│  │  - Shared across instances           │  │
│  │  - ~100-500μs latency                │  │
│  │  - TTL-based invalidation            │  │
│  └──────────────────────────────────────┘  │
│              ↓ miss                         │
│  ┌──────────────────────────────────────┐  │
│  │  L3: Backend (PostgreSQL/Sled)       │  │
│  │  - Durable, persistent               │  │
│  │  - Source of truth                   │  │
│  │  - ~1-10ms latency                   │  │
│  └──────────────────────────────────────┘  │
└─────────────────────────────────────────────┘

Features

• Write Strategies: Write-through, write-back, write-behind
• Read-through: Automatic cache population on miss
• Refresh-ahead: Proactive refresh before TTL expiry
• Statistics: Per-layer hit rates, latency, evictions
• Bulk Operations: Optimized batch get/put operations
• Cache Warming: Preload hot keys on startup

Example Usage

use processor::state::{
    CachedStateBackend, CacheConfig, WriteStrategy, EvictionPolicy,
    MemoryStateBackend,
};
use std::time::Duration;
use redis::Client;

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    // Configure cache
    let config = CacheConfig {
        l1_capacity: 10_000,
        l1_ttl: Duration::from_secs(300),
        l2_ttl: Duration::from_secs(3600),
        write_strategy: WriteStrategy::WriteThrough,
        eviction_policy: EvictionPolicy::LRU,
        enable_stats: true,
    };

    // Create backend
    let base_backend = MemoryStateBackend::new(None);
    let redis_client = Client::open("redis://127.0.0.1/")?;

    let cached_backend = CachedStateBackend::new(
        base_backend,
        Some(redis_client),
        config,
    ).await?;

    // Use it like any StateBackend
    cached_backend.put(b"key1", b"value1").await?;
    let value = cached_backend.get(b"key1").await?;

    // Check stats
    let stats = cached_backend.stats().await;
    println!("L1 hit rate: {:.2}%", stats.l1_hit_rate * 100.0);
    println!("L2 hit rate: {:.2}%", stats.l2_hit_rate * 100.0);

    Ok(())
}

Structs

CacheConfig

Cache configuration

CacheStats

Cache statistics

CachedStateBackend

Tiered caching state backend

Enums

EvictionPolicy

Cache eviction policy

WriteStrategy

Write strategy for cache updates