Module models

Transformer models.

Modules

albert

ALBERT (Lan et al., 2020)

bert

BERT (Devlin et al., 2018)

roberta

RoBERTa (Liu et al., 2018) and XLM-RoBERTa (Conneau et al., 2019).

sinusoidal

Word embeddings with sinusoidal position embeddings.

squeeze_albert

SqueezeBERT (Iandola et al., 2020) + ALBERT (Lan et al., 2020)

squeeze_bert

SqueezeBERT (Iandola et al., 2020)

Structs

HiddenLayer

Hidden layer output and attention.

Enums

LayerOutput

Output of a BERT layer.

Traits

Encoder

Encoder networks.