Struct AlbertEncoderConfig

pub struct AlbertEncoderConfig { /* private fields */ }

ALBERT encoder configuration.

Configuration for AlbertEncoder. ALBERT differs from an encoder like BERT in two ways:

1. The embedding size can be different from the hidden size. A projection matrix is used when the sizes differ.
2. A group of layers can share parameters. For instance, if the number of layers (n_hidden_layers) is 12 and the number of groups (n_hidden_groups) is 4, then layers 0..3, 3..6, 6..9, and 9..12 will share parameters.

Implementations

impl AlbertEncoderConfig

pub fn embeddings(self, embeddings: Box<dyn BuildEmbeddings>) -> Self

Encoder embeddings.

Default: TransformerEmbeddingsConfig::default()

pub fn layer_group(self, layer_group: Box<dyn BuildEncoderLayer>) -> Self

Number of layers within a group.

Default: AlbertLayerGroupConfig::default()

pub fn n_hidden_groups(self, n_hidden_groups: usize) -> Self

Number of hidden groups.

Default: 1

pub fn n_hidden_layers(self, n_hidden_layers: usize) -> Self

Number of hidden layers.

Default: 12

Trait Implementations

impl BuildArchitecture for AlbertEncoderConfig

type Architecture = AlbertEncoder

The architecture to build.

fn build(&self, vb: VarBuilder<'_>) -> Result<Self::Architecture, BoxedError>

Build the architecture.

impl Debug for AlbertEncoderConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for AlbertEncoderConfig

fn default() -> Self

Returns the “default value” for a type.