Struct AttnConfig 

pub struct AttnConfig {
    pub num_heads: usize,
    pub num_kv_heads: usize,
    pub head_dim: usize,
    pub causal: bool,
    pub scale: f32,
    pub kv_seq_stride: usize,
    pub sliding_window: usize,
}

Configuration for attention dispatch.

Fields

num_heads: usize

num_kv_heads: usize

head_dim: usize

causal: bool

scale: f32

kv_seq_stride: usize

Stride (in rows) between head blocks in the KV buffer. 0 means contiguous (use kv_len, legacy behaviour). Set to cache_capacity when flashing against a pre-allocated cache that only has kv_len valid slots out of cache_capacity.

sliding_window: usize

Sliding-window attention size (Mistral v0.1, Gemma). 0 = disabled (full causal attention). w > 0 = each query position attends to the previous w KV positions (still bounded by causal + pos_offset + qi + 1 as the upper end).

Trait Implementations

impl Clone for AttnConfig

fn clone(&self) -> AttnConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for AttnConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for AttnConfig

fn default() -> Self

Returns the “default value” for a type.