Enum FfnAct 

pub enum FfnAct {
    Silu,
    GeluErf,
}

The gated FFN’s gate-branch activation: act(g)·u feeds down_proj. One enum, so every forward and every hand-rolled backward in the trainer names the same function instead of spelling SiLU inline.

Variants

Silu

GeluErf

Exact erf GELU (Spark-X2.5).

Implementations

impl FfnAct

pub fn from_arch(name: &str) -> Option<Self>

None for an activation the backward does not differentiate yet.

pub fn fwd<F: Fp>(self, x: F) -> F

pub fn bwd<F: Fp>(self, x: F) -> F

Trait Implementations

impl Clone for FfnAct

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for FfnAct

impl Debug for FfnAct

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for FfnAct

impl PartialEq for FfnAct

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for FfnAct