Crate llm_config_cache

Crate llm_config_cache 

Source
Expand description

Multi-tier caching for LLM Config Manager

This module provides a two-tier caching system:

  • L1 Cache: In-memory cache for ultra-fast access (LRU eviction)
  • L2 Cache: Persistent cache for warm restarts

§Performance Characteristics

  • L1 Cache: <1μs latency
  • L2 Cache: <1ms latency
  • Cache miss: 5-10ms (disk read)

Re-exports§

pub use l1::L1Cache;
pub use l2::L2Cache;
pub use manager::CacheManager;

Modules§

l1
L1 in-memory cache with LRU eviction
l2
L2 persistent cache for warm restarts
manager
Cache manager coordinating L1 and L2 caches

Enums§

CacheError

Type Aliases§

Result