Struct DispatchKey 

pub struct DispatchKey {
    pub arch: SmArch,
    pub dtype: DType,
    pub head_dim: u32,
    pub causal: bool,
    pub varlen: bool,
    pub sliding_window: Option<u32>,
    pub alibi: bool,
    pub sink: u32,
    pub paged: bool,
    pub gqa_ratio: u32,
}

Cell key for the FlashAttention dispatch table.

Every field directly affects the generated CUDA C++ template instantiation — flipping any one of them changes the resulting cubin. The table refuses to resolve unsupported combinations (e.g. fp8 on Sm80, head_dim > 256).

Fields

arch: SmArch

Target SM architecture.

dtype: DType

Element type for Q/K/V.

head_dim: u32

Per-head dimension (D). Supported: 64, 80, 96, 128, 192, 256.

causal: bool

Causal masking — autoregressive attention.

varlen: bool

Variable-length (cu_seqlens). When false, batched attention with uniform seqlen.

sliding_window: Option<u32>

Sliding-window size; None means full attention. Window size is the number of past tokens each query attends to.

alibi: bool

ALiBi linear-position biases.

sink: u32

Number of “sink” tokens (StreamingLLM); each query unconditionally attends to the first sink keys regardless of sliding_window.

paged: bool

vLLM-style paged KV-cache.

gqa_ratio: u32

Q heads per KV head. 1 = MHA, >1 = GQA, equal to num_heads = MQA.

Implementations

impl DispatchKey

pub fn validate_fwd(&self) -> Result<(), DispatchError>

Validate the cell for a forward path. Returns Err for unreachable combinations.

pub fn validate_bwd(&self) -> Result<(), DispatchError>

Validate the cell for a backward path. Currently the same as forward, but kept distinct so we can refuse e.g. fp8 backward (numerically too lossy in the stock FA3) without affecting the forward whitelist.

pub fn validate_paged(&self) -> Result<(), DispatchError>

Validate the cell for a paged forward path.

pub fn stable_hash(&self) -> u64

Stable 64-bit hash of the key. Useful as a cubin-cache index alongside the kernel-name string.

pub fn kernel_name(&self) -> String

Build the canonical mangled kernel-name expression. Mirrors the FA2/FA3 csrc naming convention so we can resolve it via NVRTC’s nvrtcGetLoweredName.

Trait Implementations

impl Clone for DispatchKey

fn clone(&self) -> DispatchKey

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for DispatchKey

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Hash for DispatchKey

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl PartialEq for DispatchKey

fn eq(&self, other: &DispatchKey) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Copy for DispatchKey

impl Eq for DispatchKey

impl StructuralPartialEq for DispatchKey