Struct SlidingWindow 

pub struct SlidingWindow {
    pub window: usize,
    pub chunk: usize,
    pub pattern: Option<usize>,
}

Sliding-window attention, which caps how many positions a layer ever keeps.

Fields

window: usize

Positions a query may attend back over.

chunk: usize

Prefill chunk size. A chunk of chunk tokens is processed against one cache state, so the first token of the chunk still needs its whole window live while the last one is being processed: window + chunk - 1 positions, not window.

pattern: Option<usize>

Gemma 2+/3 alternating pattern (ModelConfig::swa_pattern): layer il is sliding iff (il + 1) % period != 0, so every period-th layer keeps the full context. None means every layer slides.

Implementations

impl SlidingWindow

pub fn resident_positions(&self, tokens: usize) -> usize

Positions a sliding layer keeps once the sequence is long enough to saturate it.

Trait Implementations

impl Clone for SlidingWindow

fn clone(&self) -> SlidingWindow

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for SlidingWindow

impl Debug for SlidingWindow

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for SlidingWindow

impl PartialEq for SlidingWindow

fn eq(&self, other: &SlidingWindow) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for SlidingWindow