Struct TpKvRankAllocationShape 

pub struct TpKvRankAllocationShape {
    pub kv_dim_k: usize,
    pub kv_dim_v: usize,
    pub k_token_bytes: usize,
    pub v_token_bytes: usize,
    pub fixed_bytes: usize,
}

Exact allocation geometry for one rank of a tensor-parallel KV sidecar.

The context-linear coefficient and fixed allocation bytes are shared by the CUDA allocator and serving admission. Keeping both consumers on this function prevents a new KV format or rank width from making the pre-admit estimate disagree with the buffers allocated at decode.

Fields

kv_dim_k: usize

kv_dim_v: usize

k_token_bytes: usize

v_token_bytes: usize

fixed_bytes: usize

Implementations

impl TpKvRankAllocationShape

pub fn bytes_per_token(self) -> usize

pub fn allocation_bytes(self, capacity: usize) -> usize

Trait Implementations

impl Clone for TpKvRankAllocationShape

fn clone(&self) -> TpKvRankAllocationShape

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for TpKvRankAllocationShape

impl Debug for TpKvRankAllocationShape

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl Eq for TpKvRankAllocationShape

impl PartialEq for TpKvRankAllocationShape

fn eq(&self, other: &TpKvRankAllocationShape) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for TpKvRankAllocationShape