Struct MultiHeadAttention 

pub struct MultiHeadAttention { /* private fields */ }

Multi-head attention layer.

Implementations

impl MultiHeadAttention

pub fn new( name: &str, wq: Box<dyn ArrayProtocol>, wk: Box<dyn ArrayProtocol>, wv: Box<dyn ArrayProtocol>, wo: Box<dyn ArrayProtocol>, num_heads: usize, dmodel: usize, ) -> Self

Create a new multi-head attention layer.

pub fn with_params(name: &str, num_heads: usize, dmodel: usize) -> Self

Create a new multi-head attention layer with randomly initialized weights.

Trait Implementations

impl Layer for MultiHeadAttention

fn layer_type(&self) -> &str

Forward pass through the layer. Get the layer type name for serialization.

fn forward( &self, inputs: &dyn ArrayProtocol, ) -> Result<Box<dyn ArrayProtocol>, OperationError>

fn parameters(&self) -> Vec<Box<dyn ArrayProtocol>>

Get the layer’s parameters.

fn parameters_mut(&mut self) -> Vec<&mut Box<dyn ArrayProtocol>>

Get mutable references to the layer’s parameters.

fn update_parameter( &mut self, name: &str, value: Box<dyn ArrayProtocol>, ) -> Result<(), OperationError>

Update a specific parameter by name

fn parameter_names(&self) -> Vec<String>

Get parameter names

fn train(&mut self)

Set the layer to training mode.

fn eval(&mut self)

Set the layer to evaluation mode.

fn is_training(&self) -> bool

Check if the layer is in training mode.

fn name(&self) -> &str

Get the layer’s name.