Struct PositionalEncoding 

pub struct PositionalEncoding {
    pub d_model: usize,
    pub max_len: usize,
    pub table: Vec<Vec<f64>>,
}

Sinusoidal positional encoding (Vaswani et al., 2017).

For each position pos and dimension i in a d_model-dimensional embedding space: PE[pos, 2i] = sin(pos / 10000^(2i/d_model)) PE[pos, 2i+1] = cos(pos / 10000^(2i/d_model))

Fields

d_model: usize

Embedding dimensionality.

max_len: usize

Maximum sequence length supported.

table: Vec<Vec<f64>>

Pre-computed encoding table: table[pos][dim].

Implementations

impl PositionalEncoding

pub fn new(d_model: usize, max_len: usize) -> Self

Build the positional encoding table up to max_len positions.

pub fn add_to_sequence(&self, embeddings: &mut Vec<Vec<f64>>)

Add positional encoding to a sequence of embeddings in-place.

embeddings[t] is a feature vector of length d_model.

pub fn get(&self, pos: usize) -> &[f64]

Return the positional encoding vector for position pos.

Trait Implementations

impl Clone for PositionalEncoding

fn clone(&self) -> PositionalEncoding

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for PositionalEncoding

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.