Struct FlashAttnVecParams 

pub struct FlashAttnVecParams {
    pub num_heads: u32,
    pub num_kv_heads: u32,
    pub head_dim: u32,
    pub kv_seq_len: u32,
    pub kv_capacity: u32,
    pub scale: f32,
    pub mask_type: u32,
    pub sliding_window: u32,
    pub softcap: f32,
}

Parameters for the flash attention vector kernel.

Fields

num_heads: u32

Number of query attention heads.

num_kv_heads: u32

Number of key/value attention heads (GQA: may be < num_heads).

head_dim: u32

Dimension of each attention head (256 or 512).

kv_seq_len: u32

Current KV sequence length (number of valid positions).

kv_capacity: u32

KV cache capacity (stride between KV heads in positions).

scale: f32

Attention score scaling factor (e.g. 1/sqrt(head_dim) or 1.0).

mask_type: u32

Mask type: 0=none, 1=causal, 2=sliding_window.

sliding_window: u32

Sliding window size (only used when mask_type == 2).

softcap: f32

Logit softcapping (0 = disabled).

Trait Implementations

impl Clone for FlashAttnVecParams

fn clone(&self) -> FlashAttnVecParams

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for FlashAttnVecParams

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Copy for FlashAttnVecParams