Struct KvShape 

pub struct KvShape {
    pub n_layers: usize,
    pub layout: KvLayout,
    pub elem: KvElem,
}

The KV shape of a whole model: enough to price any context length.

Every layer keeps every position. That is a statement about the STORES this engine allocates, not about the architectures it runs – see the module doc’s “Why a sliding window is not a saving here”, and the test that measures a real ferrox_core::cache::KvCache rather than restating this multiplication.

Fields

n_layers: usize

layout: KvLayout

elem: KvElem

Implementations

impl KvShape

pub fn from_config(config: &ModelConfig, elem: KvElem) -> Self

Reads the shape off a config.

config.sliding_window / config.swa_pattern are deliberately NOT read: they describe what attention reads, and this module prices what the store keeps. Nothing here evicts (#33), so a windowed layer costs exactly what a full-attention one does.

Always produces a KvLayout::Gqa layout, because ModelConfig describes the generic GQA decoder – the MLA stacks carry their own hyperparameters (Deepseek2Hparams, MlaConfig) and should build their shape with KvShape::mla_expanded.

pub fn mla_expanded( n_layers: usize, n_heads: usize, qk_nope_head_dim: usize, qk_rope_head_dim: usize, v_head_dim: usize, elem: KvElem, ) -> Self

The shape a ferrox MLA decoder really allocates – see KvLayout::MlaExpanded.

pub fn per_token_kv_bytes(&self) -> u64

The plan’s headline number, and the only per-token number there is: bytes one token costs across every layer. Exact for f32/f16; for the block-quantized wires it is exact whenever a layer’s per-token element count is a multiple of the 32-element block (true for every real head-dim/kv-head combination), and rounds up otherwise.

This is also the divisor KvBudget::max_context uses. There is no separate “marginal” number any more: a marginal cost below the per-token cost would mean some layer stops growing, and none does.

pub fn kv_bytes_for_tokens(&self, tokens: usize) -> u64

Bytes one request’s KV costs at tokens of context.

pub fn describe(&self) -> String

The sentence a user should be able to read and reproduce with a calculator.

Trait Implementations

impl Clone for KvShape

fn clone(&self) -> KvShape

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for KvShape

impl Debug for KvShape

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for KvShape

impl PartialEq for KvShape

fn eq(&self, other: &KvShape) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for KvShape