Struct LeakyReLU

pub struct LeakyReLU { /* private fields */ }

Leaky Rectified Linear Unit (LeakyReLU) activation function.

The LeakyReLU function is similar to ReLU but allows a small, non-zero gradient when the unit is not active: f(x) = max(alphax, x) or f(x) = x if x > 0, alphax otherwise

Examples

use scirs2_neural::activations::LeakyReLU;
use scirs2_neural::activations::Activation;
use ndarray::Array;

let leaky_relu = LeakyReLU::new(0.01);
let input = Array::from_vec(vec![1.0, -1.0, 2.0, -2.0]).into_dyn();
let output = leaky_relu.forward(&input).unwrap();
assert_eq!(output, Array::from_vec(vec![1.0, -0.01, 2.0, -0.02]).into_dyn());

Implementations

impl LeakyReLU

pub fn new(alpha: f64) -> Self

Create a new LeakyReLU activation function with given alpha.

Trait Implementations

impl<F: Float + Debug> Activation<F> for LeakyReLU

fn forward(&self, input: &Array<F, IxDyn>) -> Result<Array<F, IxDyn>>

Apply the activation function to the input

fn backward( &self, grad_output: &Array<F, IxDyn>, output: &Array<F, IxDyn>, ) -> Result<Array<F, IxDyn>>

Compute the derivative of the activation function with respect to the input

impl Clone for LeakyReLU

fn clone(&self) -> LeakyReLU

Returns a duplicate of the value.

const fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for LeakyReLU

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for LeakyReLU

fn default() -> Self

Returns the “default value” for a type.

impl Copy for LeakyReLU