Struct AttentionModuleOptions 

pub struct AttentionModuleOptions {
    pub scale: Option<f64>,
    pub softcap: Option<f64>,
    pub is_causal: bool,
    pub causal_alignment: CausalAlignment,
}

Options for the attention module.

Fields

scale: Option<f64>

Custom scale factor applied to QK^T. When None, defaults to 1/sqrt(head_dim).

softcap: Option<f64>

Soft capping applied before softmax: softcap * tanh(scores / softcap). Used by Gemma-2 and similar models. Must be positive when set.

is_causal: bool

When true, applies causal (autoregressive) masking so that each query position can only attend to key positions at or before it. This is more efficient than passing an explicit lower-triangular bool mask because backends can use optimized kernel paths (e.g. flash attention with causal mode).

When seq_q != seq_k, causal_alignment decides which query row lines up with which key column.

causal_alignment: CausalAlignment

Where the causal diagonal is anchored when seq_q != seq_k. Ignored unless is_causal is set. Both alignments agree when seq_q == seq_k.

Options serialized before this field existed deserialize as BottomRight, the alignment burn used until then.

Trait Implementations

impl Clone for AttentionModuleOptions

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for AttentionModuleOptions

impl Debug for AttentionModuleOptions

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for AttentionModuleOptions

fn default() -> Self

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for AttentionModuleOptions

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl PartialEq for AttentionModuleOptions

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl Serialize for AttentionModuleOptions

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl StructuralPartialEq for AttentionModuleOptions