pub enum ForwardOp {
Embed {
vocab_size: u32,
hidden_dim: u32,
},
RmsNorm {
hidden_dim: u32,
eps: f32,
},
Attention {
num_heads: u32,
num_kv_heads: u32,
hidden_dim: u32,
head_dim: u32,
},
FfnSwiGlu {
hidden_dim: u32,
ffn_dim: u32,
},
LinearOut {
hidden_dim: u32,
vocab_size: u32,
tied: bool,
},
Softmax {
dim: i32,
},
Sampling {
temperature: f32,
top_k: u32,
},
ApplyRope {
head_dim: u32,
base: f32,
pos: usize,
},
}Expand description
A single operation in the transformer forward pass.
Variants§
Embed
Token → hidden state lookup via an embedding table.
RmsNorm
RMS Layer Normalisation: y = x / rms(x) * gamma.
Attention
Scaled dot-product attention with optional grouped-query (GQA).
FfnSwiGlu
SwiGLU feed-forward block: FFN(x) = down(silu(gate(x)) * up(x)).
LinearOut
Final vocabulary projection (lm_head). tied = true for Qwen2/Gemma2
where the embedding table and lm_head share weights.
Softmax
Softmax over the vocab dimension.
Sampling
Sample the next token from a probability distribution.
ApplyRope
Apply Rotary Position Embedding to Q and K projections.
pos is a placeholder (0) in the template; the runtime substitutes the
real decode position from kv_cache.seq_len at forward-pass time.
Trait Implementations§
Auto Trait Implementations§
impl Freeze for ForwardOp
impl RefUnwindSafe for ForwardOp
impl Send for ForwardOp
impl Sync for ForwardOp
impl Unpin for ForwardOp
impl UnsafeUnpin for ForwardOp
impl UnwindSafe for ForwardOp
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more