Struct SinusoidalPositionalEncoding

pub struct SinusoidalPositionalEncoding<F: Float + Debug> { /* private fields */ }

Sinusoidal positional encoding from the “Attention Is All You Need” paper

Uses sine and cosine functions of different frequencies to encode position. The advantage is that this can extrapolate to longer sequences than those seen during training.

Implementations

impl<F: Float + Debug + 'static> SinusoidalPositionalEncoding<F>

pub fn new(max_len: usize, d_model: usize) -> Result<Self>

Create a new sinusoidal positional encoding

Arguments

• max_len - Maximum sequence length
• d_model - Model embedding dimension

Returns

• A new sinusoidal positional encoding

Trait Implementations

impl<F: Clone + Float + Debug> Clone for SinusoidalPositionalEncoding<F>

fn clone(&self) -> SinusoidalPositionalEncoding<F>

Returns a duplicate of the value.

const fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<F: Debug + Float + Debug> Debug for SinusoidalPositionalEncoding<F>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<F: Float + Debug + 'static> PositionalEncoding<F> for SinusoidalPositionalEncoding<F>

fn set_training(&mut self, _training: bool)

Set training mode (does nothing for sinusoidal encoding)

fn is_training(&self) -> bool

Get training mode (always false for sinusoidal encoding)

fn forward(&self, embeddings: &Array<F, IxDyn>) -> Result<Array<F, IxDyn>>

Apply positional encoding to input embeddings

fn get_encoding(&self, seq_len: usize) -> Result<Array<F, IxDyn>>

Get the positional encoding matrix directly

fn update(&mut self, _learning_rate: F) -> Result<()>

Update learnable parameters if any

fn params(&self) -> Vec<Array<F, IxDyn>>

Get learnable parameters if any