Expand description
§Cache Eviction Policies & Memory Pressure Management
Provides intelligent sequence eviction strategies (LRU) when the physical KV-cache block pool encounters high memory pressure.
Structs§
- LruEviction
Policy - Least-Recently-Used (LRU) eviction policy tracking access timestamps.
Traits§
- Eviction
Policy - Abstract interface for sequence cache eviction policies.