Enum KvBlockLayout 

pub enum KvBlockLayout {
    UniversalTP,
    UniversalPP,
    OperationalHND,
    OperationalNHD,
    Custom([BlockDim; 4]),
    Unknown,
}

Block layout defined by dimension ordering.

Describes how the 4 permutable dimensions (layer, outer, page, head) are ordered within a fully contiguous block. The head dimension (hd) is always innermost and implicit.

The order specifies outer-to-inner dimensions, with head_dim always last.

Examples

• UniversalTP: [nh, nl, no, nt, hd] - heads outermost for TP resharding
• OperationalNHD: [nl, no, nt, nh, hd] - inner is [nt, nh, hd]

Variants

UniversalTP

Universal format: [nh, nl, no, nt, hd]

Heads are outermost to enable tensor parallelism (TP) resharding. Cache saved from one TP configuration can be loaded into another by simply slicing the head dimension differently.

UniversalPP

Pipeline parallelism format: [nl, nh, no, nt, hd]

Layers are outermost for pipeline parallelism scenarios.

OperationalHND

Operational HND format: [nl, no, nh, nt, hd]

Inner tensor shape is [nh, nt, hd] (heads, tokens, head_dim).

OperationalNHD

Operational NHD format: [nl, no, nt, nh, hd]

Inner tensor shape is [nt, nh, hd] (tokens, heads, head_dim). This is the most common format used by vLLM and other frameworks.

Custom([BlockDim; 4])

Custom ordering with explicit dimension list.

The array specifies dimensions from outermost to innermost, with head_dim always implicitly last.

Unknown

Unknown layout - fallback when format cannot be determined.

Operations involving Unknown layouts may fail or require explicit configuration.

Implementations

impl KvBlockLayout

pub fn dim_order(&self) -> Option<[BlockDim; 4]>

Get the dimension ordering as an array.

Returns the 4 dimensions from outermost to innermost. Head dimension (hd) is implicit as the innermost dimension.

Returns

None for Unknown layout, Some([BlockDim; 4]) otherwise.

pub fn requires_transform(&self, other: &Self) -> bool

Check if two layouts require transformation (not just copy).

Returns true if the layouts have different dimension orderings, meaning a transformation kernel is needed rather than a simple copy.

For Unknown→Unknown comparisons, returns false (compatible) but emits a warning so these cases can be tracked and fixed.

Returns true if one is Unknown and the other is Known (conservative).

pub fn is_operational(&self) -> bool

Check if this is an operational layout (NHD or HND).

Operational layouts are used for direct computation and have layer/outer as the outermost dimensions.

pub fn is_universal(&self) -> bool

Check if this is a universal layout (TP or PP).

Universal layouts are optimized for storage and transfer, with different parallelism-friendly orderings.

pub fn name(&self) -> &'static str

Get the layout name as a string identifier.

Trait Implementations

impl Clone for KvBlockLayout

fn clone(&self) -> KvBlockLayout

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for KvBlockLayout

impl Debug for KvBlockLayout

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for KvBlockLayout

fn default() -> KvBlockLayout

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for KvBlockLayout

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Display for KvBlockLayout

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for KvBlockLayout

impl Hash for KvBlockLayout

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl PartialEq for KvBlockLayout

fn eq(&self, other: &KvBlockLayout) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl Serialize for KvBlockLayout

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl StructuralPartialEq for KvBlockLayout