Struct EncoderModel 

pub struct EncoderModel {
    pub config: TransformerConfig,
    pub embed_tokens: Embedding,
    pub position_embeddings: LearnedPositionEmbedding,
    pub token_type_embeddings: Option<Embedding>,
    pub embeddings_layernorm: LayerNorm,
    pub layers: Vec<EncoderBlock>,
}

Complete encoder model (BERT/RoBERTa/CodeBERT).

Fields

config: TransformerConfig

Configuration

embed_tokens: Embedding

Token embedding

position_embeddings: LearnedPositionEmbedding

Position embedding (learned absolute)

token_type_embeddings: Option<Embedding>

Token type embedding (segment A/B, optional for RoBERTa but present in weights)

embeddings_layernorm: LayerNorm

Post-embedding LayerNorm

layers: Vec<EncoderBlock>

Encoder layers

Implementations

impl EncoderModel

pub fn new(config: &TransformerConfig) -> Self

Create new encoder with default initialization.

pub fn from_params( config: &TransformerConfig, params: &HashMap<String, Tensor>, ) -> Option<Self>

Create encoder from pre-trained parameters (after RoBERTa name mapping).

Expected parameter names:

• encoder.embed_tokens.weight
• encoder.position_embeddings.weight
• encoder.token_type_embeddings.weight (optional)
• encoder.embeddings_layernorm.{weight,bias}
• encoder.layers.{i}.*

pub fn from_safetensors( config: &TransformerConfig, model_path: &Path, ) -> Result<Self>

Load encoder from SafeTensors file(s).

pub fn forward(&self, token_ids: &[u32]) -> Tensor

Forward pass: token_ids → hidden states [seq_len × hidden_size].

Arguments

• token_ids - Input token IDs

Returns

Hidden states tensor (seq_len * hidden_size, flattened)

pub fn cls_embedding(&self, token_ids: &[u32]) -> Tensor

Extract [CLS] embedding (position 0) from hidden states.

For classification, the [CLS] token at position 0 attends bidirectionally to all other tokens, making it a summary representation.

pub fn num_parameters(&self) -> usize

Get total parameter count.