Type Definition dfdx::nn::modules::TransformerEncoder

pub type TransformerEncoder<const MODEL_DIM: usize, const NUM_HEADS: usize, const FF_DIM: usize, const NUM_LAYERS: usize, E, D> = Repeated<TransformerEncoderBlock<MODEL_DIM, NUM_HEADS, FF_DIM, E, D>, NUM_LAYERS>;

A transformer encoder.

Generics

• MODEL_DIM: The size of query/key/value tensors. Given to MultiHeadAttention.
• NUM_HEADS: The number of heads in MultiHeadAttention.
• FF_DIM: The size of the hidden layer in the feedforward network in TransformerEncoderBlock.
• NUM_LAYERS: The number of TransformerEncoderBlock to use.