Module text_encoder 

Text transformer encoder.

Encodes token sequences into fixed-size L2-normalised embeddings using a 12-layer bidirectional transformer with masked mean-pooling.

Architecture

token_ids (B, L)
  → TokenEmbedding(vocab_size, D) + PositionalEmbedding(max_len, D)
  → Dropout
  → [EncoderBlock × depth]
  → LayerNorm
  → MaskedMeanPool  → (B, D)
  → Linear(D, out)  → (B, D)
  → L2-normalise    → (B, D)

Structs

TextEncoder

Bidirectional transformer text encoder.

TextEncoderRecord

The record type for the module.

TextEncoderRecordItem

The record item type for the module.