Struct KvCacheAppendDescriptor 

pub struct KvCacheAppendDescriptor {
    pub batch_size: i32,
    pub num_heads: i32,
    pub new_len: i32,
    pub max_cache_len: i32,
    pub d_k: i32,
    pub d_v: i32,
    pub element: ElementKind,
}

Descriptor for a KV-cache append op.

Fields

batch_size: i32

Batch size (B).

num_heads: i32

Number of attention heads (H).

new_len: i32

Number of new K/V rows to append per sample (L_new).

max_cache_len: i32

Capacity of the cache along the sequence axis (L_max).

d_k: i32

Head dimension of K (D_k).

d_v: i32

Head dimension of V (D_v). May differ from d_k.

element: ElementKind

Element type — must match the plan’s type parameter.

Trait Implementations

impl Clone for KvCacheAppendDescriptor

fn clone(&self) -> KvCacheAppendDescriptor

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for KvCacheAppendDescriptor

impl Debug for KvCacheAppendDescriptor

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.