Struct SpilloverManager 

pub struct SpilloverManager { /* private fields */ }

Async spillover manager for write buffers

Problem Addressed

Current WriteBufferManager blocks writes when hard limit is exceeded. This causes latency spikes for foreground operations.

Solution

Add async spillover to secondary storage (disk) before blocking:

1. When soft limit exceeded → trigger async flush to SSTable
2. When 90% limit exceeded → trigger spillover to temp file
3. Only block at 100% when spillover buffer is also full

Architecture

Memtable (hot data)
    │
    ├── Soft limit (80%) → Async SSTable flush
    │
    ├── Spillover limit (90%) → Async temp file write
    │
    └── Hard limit (100%) → Block (last resort)

Spillover files are replayed into new SSTables during quiet periods.

Implementations

impl SpilloverManager

pub fn new( write_buffer: Arc<WriteBufferManager>, spillover_capacity: u64, ) -> (Self, Receiver<SpilloverRequest>)

Create a new spillover manager

pub fn should_spillover(&self) -> bool

Check if spillover should be triggered

pub fn is_spillover_full(&self) -> bool

Check if spillover buffer is full

pub fn reserve_memory( &self, bytes: u64, data: Vec<(Vec<u8>, Vec<u8>)>, ) -> Result<bool, SpilloverRequest>

Reserve memory with spillover fallback

Returns:

• Ok(false) - Memory reserved, no action needed
• Ok(true) - Memory reserved, flush should be triggered
• Err(SpilloverRequest) - Caller should spill this data before proceeding

pub fn release_spillover(&self, bytes: u64)

Release spillover capacity after recovery

pub fn is_spillover_active(&self) -> bool

Check if spillover is active

pub fn spillover_usage(&self) -> u64

Get spillover usage

pub fn spillover_capacity(&self) -> u64

Get spillover capacity

pub fn stats(&self) -> &SpilloverStats

Get statistics