pub struct TransformerBlock {
pub attn_wq: Tensor,
pub attn_wk: Tensor,
pub attn_wv: Tensor,
pub attn_wo: Tensor,
pub ff_w1: Linear,
pub ff_w2: Linear,
pub ln1: LayerNorm,
pub ln2: LayerNorm,
pub d_model: usize,
pub num_heads: usize,
}Expand description
Transformer encoder block.
Fields§
§attn_wq: TensorQuery projection weight.
attn_wk: TensorKey projection weight.
attn_wv: TensorValue projection weight.
attn_wo: TensorOutput projection weight.
ff_w1: LinearFirst feed-forward layer.
ff_w2: LinearSecond feed-forward layer.
ln1: LayerNormFirst layer norm (pre-attention).
ln2: LayerNormSecond layer norm (pre-FFN).
d_model: usizeModel width.
num_heads: usizeNumber of attention heads.
Implementations§
Auto Trait Implementations§
impl Freeze for TransformerBlock
impl RefUnwindSafe for TransformerBlock
impl Send for TransformerBlock
impl Sync for TransformerBlock
impl Unpin for TransformerBlock
impl UnsafeUnpin for TransformerBlock
impl UnwindSafe for TransformerBlock
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more