Trait Attention

pub trait Attention<T> {
    // Required method
    fn attention(&self, query: T, key: T, value: T) -> T;
}

Required Methods

fn attention(&self, query: T, key: T, value: T) -> T

Computes the attention scores using the dot product of the query and key vectors, scales them by the square root of the dimension of the key vectors, and applies a softmax function to obtain the attention weights.

Implementors