Struct CrossAttention 

pub struct CrossAttention {
    pub w_q: Tensor,
    pub w_k: Tensor,
    pub w_v: Tensor,
    pub w_o: Tensor,
    pub num_heads: usize,
    pub d_k: usize,
    pub d_model: usize,
}

Cross-attention: query from decoder, key/value from encoder output.

Stores separate projection matrices for Q (applied to decoder state) and K/V (applied to encoder memory).

Fields

w_q: Tensor

w_k: Tensor

w_v: Tensor

w_o: Tensor

num_heads: usize

d_k: usize

d_model: usize

Implementations

impl CrossAttention

pub fn new(d_model: usize, num_heads: usize) -> Result<Self, ModelError>

pub fn forward(&self, query: &Tensor, kv: &Tensor) -> Result<Tensor, ModelError>

Forward pass.

query: [seq_q, d_model] — decoder state. kv: [seq_kv, d_model] — encoder memory.

Returns [seq_q, d_model].