Struct LinearAttention 

pub struct LinearAttention { /* private fields */ }

Linear attention with random feature maps

Uses kernel trick to achieve O(n * k * d) complexity instead of O(n² * d).

Implementations

impl LinearAttention

pub fn new(dim: usize, num_features: usize) -> Self

Create new linear attention

pub fn with_kernel(dim: usize, num_features: usize, kernel: KernelType) -> Self

Create with specific kernel type

Trait Implementations

impl Attention for LinearAttention

fn compute( &self, query: &[f32], keys: &[&[f32]], values: &[&[f32]], ) -> AttentionResult<Vec<f32>>

Computes attention over the given query, keys, and values.

fn compute_with_mask( &self, query: &[f32], keys: &[&[f32]], values: &[&[f32]], mask: Option<&[bool]>, ) -> AttentionResult<Vec<f32>>

Computes attention with optional mask.

fn dim(&self) -> usize

Returns the model dimension.

fn num_heads(&self) -> usize

Returns the number of attention heads (1 for single-head attention).