Struct TransformerEncoderLayer 

pub struct TransformerEncoderLayer { /* private fields */ }

Transformer Encoder Layer.

Consists of self-attention followed by a feed-forward network, with residual connections and layer normalization.

x = x + Dropout(SelfAttention(LayerNorm(x)))
x = x + Dropout(FFN(LayerNorm(x)))

Implementations

impl TransformerEncoderLayer

pub fn new( d_model: usize, nhead: usize, dim_feedforward: usize, ) -> TransformerEncoderLayer

Create a new Transformer Encoder Layer.

Arguments

• d_model - Dimension of the model
• nhead - Number of attention heads
• dim_feedforward - Dimension of the feedforward network (typically 4 * d_model)

pub fn with_dropout(self, dropout: f32) -> TransformerEncoderLayer

Set dropout probability.

pub fn forward_with_mask( &self, src: &Tensor, src_mask: Option<&Tensor>, ) -> Tensor

Forward with optional attention mask.

Trait Implementations

impl Debug for TransformerEncoderLayer

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl Module for TransformerEncoderLayer

fn forward(&self, input: &Tensor) -> Tensor

Perform forward computation.

fn parameters(&self) -> Vec<&Tensor>

Get references to all learnable parameters.

fn parameters_mut(&mut self) -> Vec<&mut Tensor>

Get mutable references to all learnable parameters.

fn train(&mut self)

Set the module to training mode.

fn eval(&mut self)

Set the module to evaluation mode.

fn training(&self) -> bool

Check if the module is in training mode.

fn refresh_caches(&mut self)

Refresh any cached computations after parameters have been modified.

fn zero_grad(&mut self)

Zero out gradients for all parameters.

fn num_parameters(&self) -> usize

Get the number of learnable parameters.