Enum PositionEncoding 

#[non_exhaustive]
pub enum PositionEncoding {
    Sinusoidal,
    Learned,
    RoPE,
    ALiBi,
    NoPE,
}

Position-encoding strategies supported by attention layers in this crate.

Variants (Non-exhaustive)

Non-exhaustive enums could have additional variants added in future. Therefore, when matching against variants of non-exhaustive enums, an extra wildcard arm must be added to account for any future variants.

Sinusoidal

Classic sinusoidal position embeddings (Vaswani et al., 2017).

Learned

Learned absolute position embeddings (BERT, GPT-2 style).

RoPE

Rotary Position Embedding — RoPE (Su et al., 2021).

ALiBi

Attention with Linear Biases — ALiBi (Press et al., 2021).

NoPE

No position encoding (NoPE) — relies on relative information in the input or is provided externally.

Trait Implementations

impl Clone for PositionEncoding

fn clone(&self) -> PositionEncoding

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for PositionEncoding

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for PositionEncoding

fn eq(&self, other: &PositionEncoding) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Copy for PositionEncoding

impl Eq for PositionEncoding

impl StructuralPartialEq for PositionEncoding