Struct MultiScaleRetention 

pub struct MultiScaleRetention { /* private fields */ }

Multi-Scale Retention (MSR) Module

Implements retention mechanism with multiple heads, each operating at different scales

Implementations

impl MultiScaleRetention

pub fn new(config: RetNetConfig) -> CoreResult<Self>

Create a new multi-scale retention module

pub fn step( &self, input: &Array1<f32>, state: &mut Array3<f32>, ) -> CoreResult<Array1<f32>>

Recurrent forward step (O(1) inference)

Updates retention state and computes output State: (num_heads, head_dim, head_dim)

pub fn forward_sequence(&self, input: &Array2<f32>) -> CoreResult<Array2<f32>>

Parallel forward for sequence (training mode)

Input shape: (seq_len, hidden_dim) Output shape: (seq_len, hidden_dim)

pub fn reset_state(&self) -> Array3<f32>

Reset retention state

pub fn num_parameters(&self) -> usize

Get number of parameters

Trait Implementations

impl Debug for MultiScaleRetention

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.