Struct EncoderLayer 

pub struct EncoderLayer {
    pub config: EncoderLayerConfig,
    pub attention: MultiHeadAttention,
    pub ffn: FeedForward,
    pub norm1: LayerNorm,
    pub norm2: LayerNorm,
}

Transformer encoder layer

Fields

config: EncoderLayerConfig

Configuration

attention: MultiHeadAttention

Multi-head attention

ffn: FeedForward

Feed-forward network

norm1: LayerNorm

First layer normalization

norm2: LayerNorm

Second layer normalization

Implementations

impl EncoderLayer

pub fn new(config: EncoderLayerConfig) -> Result<Self>

Create a new encoder layer

pub fn build_encoder_layer_graph( &self, graph: &mut EinsumGraph, ) -> Result<Vec<usize>>

Build einsum graph for encoder layer

Input tensors:

• 0: x (input) [batch, seq_len, d_model]
• 1-N: weight matrices and parameters for attention, FFN, and layer norms

Output tensors:

• output: [batch, seq_len, d_model]

Trait Implementations

impl Clone for EncoderLayer

fn clone(&self) -> EncoderLayer

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for EncoderLayer

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.