Enum BertTopology 

pub enum BertTopology {
    PostNormLayerNorm,
    PreNormRms,
}

One transformer block’s weights. Biases that llama.cpp marks TENSOR_NOT_REQUIRED are Option, so a checkpoint without them is run without them rather than with a silently fabricated zero vector. Where an encoder layer’s norms sit, and which function they are.

bert.cpp’s graph is POST-norm with LayerNorm: attention, residual, norm, FFN, residual, norm. neo-bert.cpp:59-118 and eurobert.cpp:55-114 are the other shape – RMSNorm BEFORE each block and a bare residual after it, with one final norm at the end – and they are one topology with four table columns between them, not two graphs.

Variants

PostNormLayerNorm

bert.cpp: LayerNorm with biases, after each residual.

PreNormRms

neo-bert / eurobert: RMSNorm with a weight and no bias, before each block, and one final norm.

Trait Implementations

impl Clone for BertTopology

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for BertTopology

impl Debug for BertTopology

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for BertTopology

impl PartialEq for BertTopology

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for BertTopology