pub struct BertLayer {Show 21 fields
pub wq: WeightMatrix,
pub bq: Option<Vec<f32>>,
pub wk: WeightMatrix,
pub bk: Option<Vec<f32>>,
pub wv: WeightMatrix,
pub bv: Option<Vec<f32>>,
pub wo: WeightMatrix,
pub bo: Option<Vec<f32>>,
pub qk_norm: Option<QkLayerNorm>,
pub pre_attn_norm: Option<Vec<f32>>,
pub pre_ffn_norm: Option<Vec<f32>>,
pub attn_out_norm_w: Option<Vec<f32>>,
pub attn_out_norm_b: Option<Vec<f32>>,
pub attn_norm_2: Option<(Vec<f32>, Vec<f32>)>,
pub ffn_up: WeightMatrix,
pub ffn_up_b: Option<Vec<f32>>,
pub ffn_gate: Option<WeightMatrix>,
pub ffn_down: WeightMatrix,
pub ffn_down_b: Option<Vec<f32>>,
pub layer_out_norm_w: Option<Vec<f32>>,
pub layer_out_norm_b: Option<Vec<f32>>,
}Fields§
§wq: WeightMatrix§bq: Option<Vec<f32>>§wk: WeightMatrix§bk: Option<Vec<f32>>§wv: WeightMatrix§bv: Option<Vec<f32>>§wo: WeightMatrix§bo: Option<Vec<f32>>§qk_norm: Option<QkLayerNorm>A LayerNorm over the WHOLE Q / K projection, with biases, when
the file carries one (bert.cpp:109-123; jina-bert-v2.cpp: 30-35 creates the pair optional). Not per head: the reshape at
:110 is n_embd_head * n_head wide, so one norm covers every
head’s channels together.
pre_attn_norm: Option<Vec<f32>>attn_norm / ffn_norm: the PRE-norm weights, Some exactly
for BertTopology::PreNormRms.
pre_ffn_norm: Option<Vec<f32>>§attn_out_norm_w: Option<Vec<f32>>attn_output_norm, applied after the attention residual.
Some exactly for BertTopology::PostNormLayerNorm.
attn_out_norm_b: Option<Vec<f32>>§attn_norm_2: Option<(Vec<f32>, Vec<f32>)>attn_norm_2, jina-bert-v2’s second attention norm
(bert.cpp:156-159): the LAYER INPUT is re-added and normed a
second time before the FFN reads it.
ffn_up: WeightMatrix§ffn_up_b: Option<Vec<f32>>§ffn_gate: Option<WeightMatrix>ffn_gate, present exactly for BertFfn::SwigluPar.
ffn_down: WeightMatrix§ffn_down_b: Option<Vec<f32>>§layer_out_norm_w: Option<Vec<f32>>layer_output_norm, applied after the FFN residual. Some
exactly for BertTopology::PostNormLayerNorm.
layer_out_norm_b: Option<Vec<f32>>Auto Trait Implementations§
impl Freeze for BertLayer
impl RefUnwindSafe for BertLayer
impl Send for BertLayer
impl Sync for BertLayer
impl Unpin for BertLayer
impl UnsafeUnpin for BertLayer
impl UnwindSafe for BertLayer
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more