Struct KvWindow 

pub struct KvWindow { /* private fields */ }

How many rows a contiguous windowed KV cache keeps resident, and when it drops the ones behind the window.

BlockLayout above is the same question for the block/paged tier, where the eviction unit is a block. This is the answer for ferrox_core::cache::KvCache, where the eviction unit is a row and the only thing stopping a per-token drain is arithmetic.

Why there is slack

Dropping exactly one row per push moves every remaining row down by one every single token: window * n_kv_heads * head_dim floats per layer per token, about a megabyte per token per layer on Gemma-3-4B. So the cache is allowed to run slack rows past the window and then drops slack + 1 at once, which amortises the move to roughly window / (slack + 1) rows per token.

Why this is a type and not two lines in push

The store keeps these rows and the budget (ferrox_models::kv_budget) has to price exactly what the store keeps. #33 is the record of what happens when those two are separate statements of the same rule: the budget capped a sliding layer that no store ever capped, -c auto approved a context that did not fit, and the failure arrived as an OOM. KvWindow::rows_after is the single rule; the store calls it to decide and the budget calls it to price, so there is nothing left for them to disagree about.

Implementations

impl KvWindow

pub fn new(window: usize, slack: usize) -> Option<Self>

None for a zero window, for BlockLayoutError::ZeroWindow’s reason: a query that attends to nothing is not a model.

pub fn with_default_slack(window: usize) -> Option<Self>

The slack a caller gets when it has no opinion: half a window, so the cache peaks at 1.5x the window and moves roughly two rows per token instead of window of them.

pub fn window(&self) -> usize

pub fn slack(&self) -> usize

pub fn max_rows(&self) -> usize

The most rows this window ever leaves resident.

pub fn rows_after(&self, positions: usize) -> usize

The rule. Rows resident once the sequence has consumed positions positions and the holder has evicted at every step.

Grows one per position up to window + slack, then drops back to window and climbs again, so the resident count cycles through [window, window + slack] with period slack + 1.

The invariant every reader depends on is rows_after(p) >= min(p, window): the rows kept are always at least the last window positions, which is exactly the set a windowed attention kernel reads. Everything above that is slack the kernel skips, so evicting can only ever change where a row sits, never whether it is read. That is why turning eviction on is token-identical rather than approximately so, and it is asserted in this module’s tests rather than argued here.

Trait Implementations

impl Clone for KvWindow

fn clone(&self) -> KvWindow

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for KvWindow

impl Debug for KvWindow

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for KvWindow

impl PartialEq for KvWindow

fn eq(&self, other: &KvWindow) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for KvWindow