Struct EncoderLayer 

pub struct EncoderLayer {
    pub attn_norm: Option<W>,
    pub wqkv: W,
    pub wo: W,
    pub mlp_norm: W,
    pub wi: W,
    pub mlp_wo: W,
    pub global: bool,
    pub rope_theta: f64,
}

One encoder layer.

Fields

attn_norm: Option<W>

None on layer 0, where ModernBERT skips the norm.

wqkv: W

[3d, d], rows are q then k then v.

wo: W

[d, d].

mlp_norm: W

[d].

wi: W

[2I, d], rows are the GELU input then the gate.

mlp_wo: W

[d, I].

global: bool

Global attention, or the sliding window.

rope_theta: f64

RoPE base for this layer.

Trait Implementations

impl Clone for EncoderLayer

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for EncoderLayer

impl Debug for EncoderLayer

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for EncoderLayer

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for EncoderLayer