Struct FeedForwardNetwork 

pub struct FeedForwardNetwork { /* private fields */ }

Two-layer feedforward network suitable for use as the FFN sub-layer of a transformer block.

Implementations

impl FeedForwardNetwork

pub fn new(config: FeedForwardConfig, seed: u64) -> Self

Construct a new network from config, initialising weights with He normal initialisation seeded by seed.

pub fn forward(&mut self, input: &[f64]) -> Vec<f64>

Run a single token (flat f64 slice of length input_dim) through the network and return the output vector of length output_dim.

If the input length does not match input_dim the network applies whatever it can and pads/truncates gracefully — no panic.

pub fn forward_batch(&mut self, inputs: &[Vec<f64>]) -> Vec<Vec<f64>>

Run a batch of tokens through the network.

Returns one output vector per input token; empty input yields an empty result with no panic.

pub fn linear_transform(input: &[f64], layer: &FFLayer) -> Vec<f64>

Affine transformation: output[o] = bias[o] + Σ_i weights[o][i] * input[i].

Dimension mismatches are handled gracefully: the dot-product iterates over min(in_dim, input.len()) elements and missing bias values default to 0.0.

pub fn apply_activation(&self, x: f64) -> f64

Apply the configured activation function to a single scalar.

pub fn relu(x: f64) -> f64

Rectified Linear Unit.

pub fn gelu(x: f64) -> f64

GELU using the tanh approximation (Hendrycks & Gimpel 2016):

GELU(x) ≈ 0.5 · x · (1 + tanh(√(2/π) · (x + 0.044715 · x³)))

pub fn silu(x: f64) -> f64

Sigmoid Linear Unit (Swish): x · σ(x).

pub fn sigmoid(x: f64) -> f64

Logistic sigmoid: σ(x) = 1 / (1 + e^{−x}).

pub fn init_layer( in_dim: usize, out_dim: usize, use_bias: bool, rng: &mut u64, ) -> FFLayer

Initialise a single FFLayer with He (Kaiming) normal weights.

He init draws weights from N(0, σ²) where σ = √(2 / in_dim). Bias is always zero-initialised.

pub fn next_normal(rng: &mut u64) -> f64

Draw a standard-normal sample using the xorshift64 PRNG (Marsaglia 2003) and Box-Muller transform.

Two uniform samples u1, u2 ∈ (0, 1] are generated; one standard- normal deviate is returned.

pub fn reinit_weights(&mut self)

Reinitialise both layers using the stored rng_state, effectively resetting weights to a new He-normal draw that continues from where the original seed sequence left off. Useful for experimentation without constructing a brand-new network.

pub fn stats(&self) -> &FFStats

Reference to the accumulated runtime statistics.