Enum SparsePattern 

pub enum SparsePattern {
    LocalWindow {
        window_size: usize,
    },
    BigBird {
        window_size: usize,
        num_global_tokens: usize,
        num_random_connections: usize,
        seed: u64,
    },
    Strided {
        window_size: usize,
        stride: usize,
    },
    Dense,
}

Sparse attention pattern type.

Variants

LocalWindow

Each token attends to window_size neighbors (must be odd).

Fields

window_size: usize

BigBird

BigBird: global tokens + local window + random sparse connections.

Fields

window_size: usize

num_global_tokens: usize

num_random_connections: usize

seed: u64

Strided

Strided: every stride-th token attends globally; others use local window.

Fields

window_size: usize

stride: usize

Dense

Full dense attention (baseline).

Trait Implementations

impl Clone for SparsePattern

fn clone(&self) -> SparsePattern

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for SparsePattern

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for SparsePattern

fn eq(&self, other: &SparsePattern) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for SparsePattern