Struct MultiHeadAttention 

pub struct MultiHeadAttention {
    pub w_q: Tensor,
    pub w_k: Tensor,
    pub w_v: Tensor,
    pub w_o: Tensor,
    pub num_heads: usize,
    pub d_k: usize,
}

Multi-head attention weights.

Fields

w_q: Tensor

w_k: Tensor

w_v: Tensor

w_o: Tensor

num_heads: usize

d_k: usize

Implementations

impl MultiHeadAttention

pub fn new(config: &MultiHeadAttentionConfig) -> Result<Self, ModelError>

Creates zero-initialized multi-head attention weights.

pub fn forward(&self, input: &Tensor) -> Result<Tensor, ModelError>

Forward pass: input [seq_len, d_model] -> output [seq_len, d_model].