Struct TransformerDecoderBlock 

pub struct TransformerDecoderBlock {
    pub self_attn: MultiHeadAttention,
    pub cross_attn: CrossAttention,
    pub ffn: FeedForward,
    pub ln1_gamma: Tensor,
    pub ln1_beta: Tensor,
    pub ln2_gamma: Tensor,
    pub ln2_beta: Tensor,
    pub ln3_gamma: Tensor,
    pub ln3_beta: Tensor,
    pub d_model: usize,
}

Single transformer decoder block: masked self-attention → cross-attention → FFN, each sub-layer wrapped with residual connection and layer normalization.

Fields

self_attn: MultiHeadAttention

cross_attn: CrossAttention

ffn: FeedForward

ln1_gamma: Tensor

ln1_beta: Tensor

ln2_gamma: Tensor

ln2_beta: Tensor

ln3_gamma: Tensor

ln3_beta: Tensor

d_model: usize

Implementations

impl TransformerDecoderBlock

pub fn new( d_model: usize, num_heads: usize, d_ff: usize, ) -> Result<Self, ModelError>

pub fn forward( &self, target: &Tensor, memory: &Tensor, ) -> Result<Tensor, ModelError>

Forward pass.

target: [seq_t, d_model] — decoder input / previous decoder layer output. memory: [seq_s, d_model] — encoder output.

Returns [seq_t, d_model].