Struct MultiHeadAttention 

pub struct MultiHeadAttention { /* private fields */ }

Multi-head attention mechanism

Implementations

impl MultiHeadAttention

pub fn new(embed_dim: usize, num_heads: usize) -> Result<Self, GnnError>

Create a new multi-head attention layer

Errors

Returns GnnError::LayerConfig if embed_dim is not divisible by num_heads.

pub fn forward( &self, query: &[f32], keys: &[Vec<f32>], values: &[Vec<f32>], ) -> Vec<f32>

Forward pass: compute multi-head attention

Arguments

• query - Query vector
• keys - Key vectors from neighbors
• values - Value vectors from neighbors

Returns

Attention-weighted output vector

Trait Implementations

impl Clone for MultiHeadAttention

fn clone(&self) -> MultiHeadAttention

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for MultiHeadAttention

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for MultiHeadAttention

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Serialize for MultiHeadAttention

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.