oxidized_transformers::architectures

Trait EncoderLayer

Source 
pub trait EncoderLayer {
    // Required method
    fn forward_t(
        &self,
        input: &Tensor,
        mask: &AttentionMask,
        positions: Option<&Tensor>,
        train: bool,
    ) -> Result<Tensor, BoxedError>;
}
Expand description

Trait for encoder layers.

Required Methods§

Source

fn forward_t( &self, input: &Tensor, mask: &AttentionMask, positions: Option<&Tensor>, train: bool, ) -> Result<Tensor, BoxedError>

Apply the encoder layer to the given hidden representations.

  • input - Hidden representations to apply the layer to. Shape: (batch_size, seq_len, width)
  • attention_mask - Attention mask. Sequence elements for which the corresponding mask element is set to false are ignored during attention calculation. Shape: (batch_size, seq_len)
  • positions - Input positions. Shape: (batch_size, seq_len)
  • train - Whether to train the layer.

Returns layer output and the cache. Shape: (batch_size, seq_len, width)

Implementors§