Struct TransformerDecoderLayer 

pub struct TransformerDecoderLayer { /* private fields */ }

A single Transformer decoder layer.

Consists of:

1. Masked multi-head self-attention (causal)
2. Multi-head cross-attention over encoder output
3. Position-wise feedforward network

Each sublayer has residual connections and layer normalization.

Shape

• Target: (N, T, E)
• Memory: (N, S, E)
• Output: (N, T, E)

Implementations

impl TransformerDecoderLayer

pub fn new(d_model: usize, nhead: usize, dim_feedforward: usize) -> Self

Creates a new TransformerDecoderLayer (post-norm, default).

Arguments

• d_model - Embedding dimension
• nhead - Number of attention heads
• dim_feedforward - Hidden dimension of feedforward network

pub fn new_with_pre_norm( d_model: usize, nhead: usize, dim_feedforward: usize, pre_norm: bool, ) -> Self

Creates a TransformerDecoderLayer with configurable norm ordering.

pub fn forward_with_memory( &self, tgt: &Variable, memory: &Variable, tgt_mask: Option<&Variable>, memory_mask: Option<&Variable>, ) -> Variable

Forward pass with encoder memory and optional masks.

Arguments

• tgt - Target sequence (N, T, E)
• memory - Encoder output (N, S, E)
• tgt_mask - Optional causal mask for self-attention
• memory_mask - Optional mask for cross-attention

pub fn d_model(&self) -> usize

Returns the model dimension.

Trait Implementations

impl Module for TransformerDecoderLayer

fn forward(&self, input: &Variable) -> Variable

Performs the forward pass.

fn parameters(&self) -> Vec<Parameter>

Returns all parameters of this module.

fn named_parameters(&self) -> HashMap<String, Parameter>

Returns named parameters of this module.

fn name(&self) -> &'static str

Returns the module name for debugging.

fn num_parameters(&self) -> usize

Returns the number of trainable parameters.

fn train(&mut self)

Sets the module to training mode.

fn eval(&mut self)

Sets the module to evaluation mode.

fn set_training(&mut self, _training: bool)

Sets the training mode.

fn is_training(&self) -> bool

Returns whether the module is in training mode.

fn zero_grad(&self)

Zeros all gradients of parameters.

fn to_device(&self, device: Device)

Moves all parameters to the specified device.