Struct GatedLinearUnit 

pub struct GatedLinearUnit { /* private fields */ }

Gated Linear Unit: splits input in half and applies gate

GLU(x, W, V) = (xW + b) * sigmoid(xV + c)

Implementations

impl GatedLinearUnit

pub fn new() -> Self

Create a new GLU with default sigmoid gate

pub fn swiglu() -> Self

Create with SiLU gate (SwiGLU, commonly used in modern architectures)

pub fn geglu() -> Self

Create with GELU gate (GeGLU)

pub fn forward(&self, x: &Array1<f32>) -> Array1<f32>

Apply GLU to input (input should have even dimension)

Splits input into [x, gate] and returns x * activation(gate)

Trait Implementations

impl Clone for GatedLinearUnit

fn clone(&self) -> GatedLinearUnit

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for GatedLinearUnit

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for GatedLinearUnit

fn default() -> Self

Returns the “default value” for a type.