Struct FlashAttnVecParams 

pub struct FlashAttnVecParams {
    pub num_heads: u32,
    pub num_kv_heads: u32,
    pub head_dim: u32,
    pub kv_seq_len: u32,
    pub kv_capacity: u32,
    pub scale: f32,
    pub mask_type: u32,
    pub sliding_window: u32,
    pub softcap: f32,
    pub q_seq_len: u32,
}

Parameters for the flash attention vector kernel.

Fields

num_heads: u32

Number of query attention heads.

num_kv_heads: u32

Number of key/value attention heads (GQA: may be < num_heads).

head_dim: u32

Dimension of each attention head (256 or 512).

kv_seq_len: u32

Current KV sequence length (number of valid positions).

kv_capacity: u32

KV cache capacity (stride between KV heads in positions).

scale: f32

Attention score scaling factor (e.g. 1/sqrt(head_dim) or 1.0).

mask_type: u32

Mask type: 0=none, 1=causal, 2=sliding_window.

sliding_window: u32

Sliding window size (only used when mask_type == 2).

softcap: f32

Logit softcapping (0 = disabled).

q_seq_len: u32

ADR-034 task #89 (2026-05-21) — number of queries to process per dispatch. Default 1 (decode). For spec-decode verify with cur_len > 0 + seq_len in [2, 8], call with q_seq_len = seq_len. The shader dispatches q_seq_len threadgroups in grid.x; each handles one (query, head) pair with causal mask abs_pos = kv_seq_len - q_seq_len + iq1. Peer-code reference: llama.cpp’s kernel_flash_attn_ext_vec (NQPSG=1, ne01 threadgroups in qL dim).

Implementations

impl FlashAttnVecParams

pub const DEFAULT_Q_SEQ_LEN: u32 = 1

Default q_seq_len for decode (single query). Use for all existing call sites; spec-decode verify sets q_seq_len > 1.

Trait Implementations

impl Clone for FlashAttnVecParams

fn clone(&self) -> FlashAttnVecParams

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for FlashAttnVecParams

impl Debug for FlashAttnVecParams

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.