Struct TensorActivation 

pub struct TensorActivation { /* private fields */ }

Activation layer with forward/backward support and call statistics.

Implementations

impl TensorActivation

pub fn new(config: ActivationConfig) -> Self

Create a new activation layer from the given configuration.

pub fn relu(x: f64) -> f64

ReLU activation: max(0, x).

pub fn sigmoid(x: f64) -> f64

Sigmoid activation: 1 / (1 + exp(-x)).

pub fn tanh_act(x: f64) -> f64

Tanh activation (wrapper around f64::tanh).

pub fn gelu(x: f64) -> f64

GELU activation (approximate): 0.5 * x * (1 + tanh(sqrt(2/pi) * (x + 0.044715 * x^3)))

pub fn swish(x: f64) -> f64

Swish activation: x * sigmoid(x).

pub fn softmax(input: &[f64]) -> Vec<f64>

Softmax over a slice of values, using the log-sum-exp trick for numerical stability.

pub fn forward(&mut self, input: &[f64]) -> Vec<f64>

Apply the configured activation element-wise (Softmax across all elements).

pub fn backward(&mut self, input: &[f64], grad_output: &[f64]) -> Vec<f64>

Compute the gradient of the loss w.r.t. the activation input.

• input – the original pre-activation values.
• grad_output – the upstream gradient (dL/d(output)).

Returns dL/d(input).

pub fn stats(&self) -> ActivationStats

Return runtime statistics.