Struct BertHparams 

pub struct BertHparams {

    pub arch: String,
    pub n_layer: usize,
    pub n_embd: usize,
    pub n_ff: usize,
    pub n_head: usize,
    pub n_head_kv: usize,
    pub n_ctx_train: usize,
    pub n_token_types: usize,
    pub layer_norm_eps: f32,
    pub rope_theta: Option<f32>,
    pub rope_dim: usize,
    pub ffn: BertFfn,
    pub topology: BertTopology,
    pub rope_interleaved: bool,
    pub alibi_slopes: Option<Vec<f32>>,
    pub pooling: PoolingType,
    pub cls_id: u32,
    pub sep_id: u32,
}

bert.* metadata, after the loader has checked it.

Fields

arch: String

n_layer: usize

n_embd: usize

n_ff: usize

n_head: usize

n_head_kv: usize

n_ctx_train: usize

Height of the learned position table.

n_token_types: usize

layer_norm_eps: f32

rope_theta: Option<f32>

NEOX RoPE on Q and K instead of a learned position table: Some(theta) for the architectures bert.cpp:126-133 rotates (nomic-bert, nomic-bert-moe, jina-bert-v3), None for bert itself, which adds position_embd at :90 instead.

One field for both facts on purpose: a file cannot have a position table AND a rotation here, because upstream decides both from the architecture and never reads the table for a rotating one – measured, libllama’s load log never names position_embd for nomic-bert.

rope_dim: usize

How many of each head’s channels rotate ({arch}.rope.dimension_count), head_dim when the key is absent. Only read when Self::rope_theta is Some.

ffn: BertFfn

The FFN this architecture runs (bert.cpp:179-201).

topology: BertTopology

Where this architecture’s norms sit and what they are.

rope_interleaved: bool

true for the NORM (interleaved-pair) rotation, false for NEOX (split-half). llama_model_rope_type answers NORM for neo-bert alone among the encoders here.

alibi_slopes: Option<Vec<f32>>

ALiBi slopes, one per head, for the architecture whose graph carries a positional bias instead of a table or a rotation.

jina-bert-v2.cpp:5 sets f_max_alibi_bias = 8.0f as a LITERAL and bert.cpp:78-80 builds no inp_pos for it at all, so this is the only place position enters that model. The bias is SYMMETRIC here – llama-graph.cpp:442 fills the mask with -|p0 - p1| for a non-causal model, where the decoder’s is p_key - p_query – which is why the encoder computes its own rather than calling the decoder’s row helper.

pooling: PoolingType

cls_id: u32

[CLS] / [SEP], from tokenizer.ggml.bos_token_id and tokenizer.ggml.seperator_token_id (upstream’s spelling of the key, typo included). See BertEncoder::wrap_special.

sep_id: u32

Implementations

impl BertHparams

pub fn head_dim(&self) -> usize

Trait Implementations

impl Clone for BertHparams

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for BertHparams

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.