Enum KVLayout 

pub enum KVLayout {
    Unbatched {
        num_heads: usize,
        seq_len: usize,
        head_dim: usize,
    },
    Batched {
        batch: usize,
        num_heads: usize,
        seq_len: usize,
        head_dim: usize,
    },
}

Layout metadata for compressed KV tensors.

Variants

Unbatched

Unbatched KV: [num_heads, seq_len, head_dim].

Fields

num_heads: usize

seq_len: usize

head_dim: usize

Batched

Batched KV: [batch, num_heads, seq_len, head_dim].

Fields

batch: usize

num_heads: usize

seq_len: usize

head_dim: usize

Trait Implementations

impl Clone for KVLayout

fn clone(&self) -> KVLayout

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for KVLayout

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for KVLayout

fn eq(&self, other: &KVLayout) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Copy for KVLayout

impl Eq for KVLayout

impl StructuralPartialEq for KVLayout