Struct AttentionPattern 

pub struct AttentionPattern {
    pub sliding_window: Option<usize>,
    pub pattern: usize,
    pub rope_local_theta: f64,
    pub query_pre_attn_scalar: Option<f64>,
    pub max_window_layers: Option<usize>,
}

Attention RoPE/scale settings that vary by layer type (Gemma2/3): pattern-th layers are “global” (full attention, rope_theta); the rest are “local” (sliding-window attention, rope_local_theta).

Fields

sliding_window: Option<usize>

Sliding-window span for local layers; None = all layers global.

pattern: usize

Every Nth layer is global (HF sliding_window_pattern, default 6).

rope_local_theta: f64

RoPE base frequency for local layers (rope_local_base_freq).

query_pre_attn_scalar: Option<f64>

Attention logit scale divisor (query_pre_attn_scalar); when None, the scale is 1/sqrt(head_dim).

max_window_layers: Option<usize>

Qwen2-style partition, stored raw: the first N layers are global and layers >= N slide — the inverse of pattern’s every-Nth-global. ArchSpec::resolve turns it into the per-layer layout. All shipped qwen2.5 checkpoints disable sliding anyway (use_sliding_window: false nulls sliding_window at parse).

Implementations

impl AttentionPattern

pub fn is_global_layer(&self, i: usize) -> bool

Whether layer i uses global attention (vs sliding-window local).

Trait Implementations

impl Clone for AttentionPattern

fn clone(&self) -> AttentionPattern

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for AttentionPattern

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for AttentionPattern

fn default() -> Self

Returns the “default value” for a type.