Struct AttentionParams 

pub struct AttentionParams {
    pub num_heads: usize,
    pub num_kv_heads: usize,
    pub head_dim: usize,
    pub softmax_scale: f32,
    pub causal: bool,
}

Parameters describing a single attention call.

Fields

num_heads: usize

num_kv_heads: usize

head_dim: usize

softmax_scale: f32

Softmax scale (typically 1 / sqrt(head_dim)).

causal: bool

Whether to apply a causal mask.

Trait Implementations

impl Clone for AttentionParams

fn clone(&self) -> AttentionParams

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for AttentionParams

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.