Struct SinusoidalPositionalEncoding 

pub struct SinusoidalPositionalEncoding<F: Float + Debug + NumAssign> { /* private fields */ }

Sinusoidal Positional Encoding

Implements the classic positional encoding from “Attention Is All You Need”: PE(pos, 2i) = sin(pos / 10000^(2i/d_model)) PE(pos, 2i+1) = cos(pos / 10000^(2i/d_model))

This encoding has several desirable properties:

• Deterministic and parameter-free
• Each position gets a unique encoding
• Can extrapolate to longer sequences than seen during training
• Allows the model to attend to relative positions

Implementations

impl<F: Float + Debug + NumAssign> SinusoidalPositionalEncoding<F>

pub fn new(d_model: usize, max_len: usize) -> Self

Create a new sinusoidal positional encoding

Arguments

• d_model - Model dimension (must be even)
• max_len - Maximum sequence length

pub fn with_dropout(d_model: usize, max_len: usize, dropout: F) -> Self

Create with dropout

pub fn params(&self) -> Vec<&Array<F, IxDyn>>

Get parameters (sinusoidal encoding has no trainable parameters)

pub fn set_training(&mut self, _training: bool)

Set training mode (no-op for sinusoidal encoding as it has no trainable parameters)

Trait Implementations

impl<F: Clone + Float + Debug + NumAssign> Clone for SinusoidalPositionalEncoding<F>

fn clone(&self) -> SinusoidalPositionalEncoding<F>

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<F: Debug + Float + Debug + NumAssign> Debug for SinusoidalPositionalEncoding<F>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<F: Float + Debug + NumAssign> PositionalEncoding<F> for SinusoidalPositionalEncoding<F>

fn encode(&self, seq_len: usize) -> Array2<F>

Encode positions for a sequence of given length

fn apply(&self, input: &Array3<F>) -> Result<Array3<F>>

Apply positional encoding to an input tensor

fn clone_box(&self) -> Box<dyn PositionalEncoding<F> + Send + Sync>

where F: Send + Sync + 'static,

Clone the positional encoding into a boxed trait object

fn d_model(&self) -> usize

Get the model dimension

fn max_len(&self) -> usize

Get the maximum sequence length supported

fn forward(&self, input: &Array<F, IxDyn>) -> Result<Array<F, IxDyn>>

Forward pass with dynamic array

fn update(&mut self, _learning_rate: F) -> Result<()>

Update trainable parameters (for learned encodings)