Struct SparseFfn 

pub struct SparseFfn { /* private fields */ }

Sparse Feed-Forward Network computation.

This implements a two-layer FFN that can compute using only a subset of neurons:

• W1: [hidden_dim, input_dim] - first projection (row-major for neuron access)
• W2_T: [hidden_dim, output_dim] - second projection TRANSPOSED (row-major for contiguous access)
• Activation function applied between layers

The sparse forward pass:

1. Sparse first layer: only compute active neurons
2. Apply activation function
3. Sparse second layer: accumulate only active neuron contributions (now contiguous!)

Performance Optimization

W2 is stored transposed so that accessing columns (by neuron index) becomes row access, which is contiguous in memory. This provides 15-25% speedup in the sparse accumulation step.

Implementations

impl SparseFfn

pub fn new( input_dim: usize, hidden_dim: usize, output_dim: usize, activation: ActivationType, ) -> Result<Self>

Create a new sparse FFN with given dimensions.

pub fn from_weights( w1: Array2<f32>, w2: Array2<f32>, b1: Array1<f32>, b2: Array1<f32>, activation: ActivationType, ) -> Result<Self>

Create from existing weights.

pub fn input_dim(&self) -> usize

Get input dimension.

pub fn hidden_dim(&self) -> usize

Get hidden dimension.

pub fn output_dim(&self) -> usize

Get output dimension.

pub fn forward_sparse( &self, input: &[f32], active_neurons: &[usize], ) -> Result<Vec<f32>>

Compute FFN using only active neurons (sparse computation).

This is the main optimization: only compute activations for predicted neurons.

pub fn forward_dense(&self, input: &[f32]) -> Result<Vec<f32>>

Compute FFN using all neurons (dense computation).

This is the baseline for comparison and correctness checking.

Trait Implementations

impl Clone for SparseFfn

fn clone(&self) -> SparseFfn

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for SparseFfn

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for SparseFfn

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl FeedForward for SparseFfn

fn forward_sparse( &self, input: &[f32], active_neurons: &[usize], ) -> Result<Vec<f32>>

Sparse forward pass using only active neurons.

fn forward_dense(&self, input: &[f32]) -> Result<Vec<f32>>

Dense forward pass using all neurons.

impl Serialize for SparseFfn

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.