Struct MultiHeadAttention

pub struct MultiHeadAttention { /* private fields */ }

Multi-head attention layer.

Implementations

impl MultiHeadAttention

pub fn new( name: &str, wq: Box<dyn ArrayProtocol>, wk: Box<dyn ArrayProtocol>, wv: Box<dyn ArrayProtocol>, wo: Box<dyn ArrayProtocol>, num_heads: usize, d_model: usize, ) -> Self

Create a new multi-head attention layer.

pub fn with_shape(name: &str, d_model: usize, num_heads: usize) -> Self

Create a new multi-head attention layer with randomly initialized weights.

Trait Implementations

impl Layer for MultiHeadAttention

fn forward( &self, inputs: &dyn ArrayProtocol, ) -> Result<Box<dyn ArrayProtocol>, OperationError>

Forward pass through the layer.

fn parameters(&self) -> Vec<Box<dyn ArrayProtocol>>

Get the layer’s parameters.

fn train(&mut self)

Set the layer to training mode.

fn eval(&mut self)

Set the layer to evaluation mode.

fn is_training(&self) -> bool

Check if the layer is in training mode.

fn name(&self) -> &str

Get the layer’s name.

fn as_any(&self) -> &dyn Any

Downcast the layer to Any for type-specific operations.