Struct DecoderLayer 

pub struct DecoderLayer {
    pub config: DecoderLayerConfig,
    pub self_attention: MultiHeadAttention,
    pub cross_attention: MultiHeadAttention,
    pub ffn: FeedForward,
    pub norm1: LayerNorm,
    pub norm2: LayerNorm,
    pub norm3: LayerNorm,
}

Transformer decoder layer

Fields

config: DecoderLayerConfig

Configuration

self_attention: MultiHeadAttention

Masked self-attention

cross_attention: MultiHeadAttention

Cross-attention

ffn: FeedForward

Feed-forward network

norm1: LayerNorm

First layer normalization (self-attention)

norm2: LayerNorm

Second layer normalization (cross-attention)

norm3: LayerNorm

Third layer normalization (FFN)

Implementations

impl DecoderLayer

pub fn new(config: DecoderLayerConfig) -> Result<Self>

Create a new decoder layer

pub fn build_decoder_layer_graph( &self, graph: &mut EinsumGraph, ) -> Result<Vec<usize>>

Build einsum graph for decoder layer

Input tensors:

• 0: x (decoder input) [batch, tgt_len, d_model]
• 1: encoder_output [batch, src_len, d_model]
• 2-N: weight matrices and parameters

Output tensors:

• output: [batch, tgt_len, d_model]

Trait Implementations

impl Clone for DecoderLayer

fn clone(&self) -> DecoderLayer

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for DecoderLayer

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.