Enum Activation

pub enum Activation {
    Gelu,
    GeluNew,
    Relu,
}

Variants

Gelu

GELU activation function.

GELU(x)=x Φ(x)

where Φ(x) is the CDF for the Gaussian distribution.

GeluNew

GELU activation function (Google/OpenAI flavor).

GELU(x)=x Φ(x)

where Φ(x) is the CDF for the Gaussian distribution.

Relu

ReLU activation function

ReLU(x)=max(0,x)

Trait Implementations

impl Clone for Activation

fn clone(&self) -> Activation

Returns a duplicate of the value.

const fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Activation

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for Activation

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl FallibleModule for Activation

type Error = TransformerError

The error type.

fn forward(&self, input: &Tensor) -> Result<Tensor, Self::Error>

Apply the module.

impl PartialEq for Activation

fn eq(&self, other: &Activation) -> bool

Tests for self and other values to be equal, and is used by ==.

const fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl TryFrom<&str> for Activation

type Error = TransformerError

The type returned in the event of a conversion error.

fn try_from(value: &str) -> Result<Self, Self::Error>

Performs the conversion.

impl TryFrom<String> for Activation

type Error = TransformerError

The type returned in the event of a conversion error.

fn try_from(value: String) -> Result<Self, Self::Error>

Performs the conversion.

impl Copy for Activation

impl Eq for Activation

impl StructuralPartialEq for Activation