Struct SinusoidalEmbeddings

pub struct SinusoidalEmbeddings { /* private fields */ }

Embeddings layer that uses word embeddings with sinusoidal positions.

The word embeddings in this layer can be optimized, but the sinusoidal positions are generated on-the-fly.

Implementations

impl SinusoidalEmbeddings

pub fn new<'a>( vs: impl Borrow<PathExt<'a>>, config: &impl WordEmbeddingsConfig, p_norm: Option<f64>, ) -> Result<SinusoidalEmbeddings, TransformerError>

Create piece embeddings with sinusoidal position embeddings.

If a p_norm is specified position embeddings are normalized using this norm.

Trait Implementations

impl Debug for SinusoidalEmbeddings

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl FallibleModuleT for SinusoidalEmbeddings

type Error = TransformerError

The error type.

fn forward_t( &self, input_ids: &Tensor, train: bool, ) -> Result<Tensor, Self::Error>

Apply the module.