Trait ActivationOps 

pub trait ActivationOps<B>
where
    B: Backend,
{
    // Provided methods
    fn leaky_relu(
        tensor: <B as BackendTypes>::FloatTensorPrimitive,
        negative_slope: Scalar,
    ) -> <B as BackendTypes>::FloatTensorPrimitive { ... }
    fn relu(
        tensor: <B as BackendTypes>::FloatTensorPrimitive,
    ) -> <B as BackendTypes>::FloatTensorPrimitive { ... }
    fn relu_backward(
        output: <B as BackendTypes>::FloatTensorPrimitive,
        grad: <B as BackendTypes>::FloatTensorPrimitive,
    ) -> <B as BackendTypes>::FloatTensorPrimitive { ... }
    fn gelu(
        tensor: <B as BackendTypes>::FloatTensorPrimitive,
    ) -> <B as BackendTypes>::FloatTensorPrimitive { ... }
    fn prelu(
        tensor: <B as BackendTypes>::FloatTensorPrimitive,
        alpha: <B as BackendTypes>::FloatTensorPrimitive,
    ) -> <B as BackendTypes>::FloatTensorPrimitive { ... }
    fn gelu_backward(
        x: <B as BackendTypes>::FloatTensorPrimitive,
        grad: <B as BackendTypes>::FloatTensorPrimitive,
    ) -> <B as BackendTypes>::FloatTensorPrimitive { ... }
    fn sigmoid(
        tensor: <B as BackendTypes>::FloatTensorPrimitive,
    ) -> <B as BackendTypes>::FloatTensorPrimitive { ... }
    fn sigmoid_backward(
        output: <B as BackendTypes>::FloatTensorPrimitive,
        grad: <B as BackendTypes>::FloatTensorPrimitive,
    ) -> <B as BackendTypes>::FloatTensorPrimitive { ... }
    fn hard_sigmoid(
        tensor: <B as BackendTypes>::FloatTensorPrimitive,
        alpha: Scalar,
        beta: Scalar,
    ) -> <B as BackendTypes>::FloatTensorPrimitive { ... }
    fn log_sigmoid(
        tensor: <B as BackendTypes>::FloatTensorPrimitive,
    ) -> <B as BackendTypes>::FloatTensorPrimitive { ... }
    fn softmax(
        tensor: <B as BackendTypes>::FloatTensorPrimitive,
        dim: usize,
    ) -> <B as BackendTypes>::FloatTensorPrimitive { ... }
    fn log_softmax(
        tensor: <B as BackendTypes>::FloatTensorPrimitive,
        dim: usize,
    ) -> <B as BackendTypes>::FloatTensorPrimitive { ... }
    fn softmin(
        tensor: <B as BackendTypes>::FloatTensorPrimitive,
        dim: usize,
    ) -> <B as BackendTypes>::FloatTensorPrimitive { ... }
    fn log_sigmoid_backward(
        x: <B as BackendTypes>::FloatTensorPrimitive,
        grad: <B as BackendTypes>::FloatTensorPrimitive,
    ) -> <B as BackendTypes>::FloatTensorPrimitive { ... }
}

Activation function operations.

This trait let backend implementations override activation functions for better performance.

Provided Methods

fn leaky_relu( tensor: <B as BackendTypes>::FloatTensorPrimitive, negative_slope: Scalar, ) -> <B as BackendTypes>::FloatTensorPrimitive

Applies the LeakyReLU activation function.

Arguments

• tensor - The tensor.
• negative_slope - The negative_slope value that values smaller than 0 are multiplied with.

Returns

The output tensor.

fn relu( tensor: <B as BackendTypes>::FloatTensorPrimitive, ) -> <B as BackendTypes>::FloatTensorPrimitive

Applies the ReLU activation function.

Arguments

• tensor - The tensor.

Returns

The output tensor.

fn relu_backward( output: <B as BackendTypes>::FloatTensorPrimitive, grad: <B as BackendTypes>::FloatTensorPrimitive, ) -> <B as BackendTypes>::FloatTensorPrimitive

Applies the ReLU activation function backward.

Arguments

• output - The output tensor.

Returns

The gradient.

fn gelu( tensor: <B as BackendTypes>::FloatTensorPrimitive, ) -> <B as BackendTypes>::FloatTensorPrimitive

Applies the Gelu activation function.

Arguments

• tensor - The tensor.

Returns

The output tensor.

fn prelu( tensor: <B as BackendTypes>::FloatTensorPrimitive, alpha: <B as BackendTypes>::FloatTensorPrimitive, ) -> <B as BackendTypes>::FloatTensorPrimitive

Applies the PReLu activation function.

Arguments

• tensor - The input tensor
• alpha - The weight tensor

fn gelu_backward( x: <B as BackendTypes>::FloatTensorPrimitive, grad: <B as BackendTypes>::FloatTensorPrimitive, ) -> <B as BackendTypes>::FloatTensorPrimitive

Applies the Gelu activation function backward.

Arguments

• x - The tensor.
• grad - The gradient.

Returns

The output tensor.

fn sigmoid( tensor: <B as BackendTypes>::FloatTensorPrimitive, ) -> <B as BackendTypes>::FloatTensorPrimitive

Applies the Sigmoid activation function.

Arguments

• tensor - The tensor.

Returns

The output tensor.

fn sigmoid_backward( output: <B as BackendTypes>::FloatTensorPrimitive, grad: <B as BackendTypes>::FloatTensorPrimitive, ) -> <B as BackendTypes>::FloatTensorPrimitive

Applies the Sigmoid activation function backward.

Arguments

• output - The output tensor of the sigmoid function.
• grad - The gradient.

Returns

The output tensor.

fn hard_sigmoid( tensor: <B as BackendTypes>::FloatTensorPrimitive, alpha: Scalar, beta: Scalar, ) -> <B as BackendTypes>::FloatTensorPrimitive

Applies the hard Sigmoid activation function.

Arguments

• tensor - The tensor.
• alpha - The alpha value that the tensor is multiplied with.
• beta - The beta value that is added to the tensor

Returns

The output tensor.

fn log_sigmoid( tensor: <B as BackendTypes>::FloatTensorPrimitive, ) -> <B as BackendTypes>::FloatTensorPrimitive

Applies the LogSigmoid activation function.

Arguments

• tensor - The tensor.

Returns

The output tensor.

fn softmax( tensor: <B as BackendTypes>::FloatTensorPrimitive, dim: usize, ) -> <B as BackendTypes>::FloatTensorPrimitive

Applies the softmax function along the given dimension.

Uses the max-shift trick for numerical stability: the per-row max is detached so no gradient flows back through it (the shift is a numerical-stability transformation, not part of the function).

Arguments

• tensor - The tensor.
• dim - The dimension along which softmax is computed.

Returns

The output tensor.

fn log_softmax( tensor: <B as BackendTypes>::FloatTensorPrimitive, dim: usize, ) -> <B as BackendTypes>::FloatTensorPrimitive

Applies the log-softmax function along the given dimension.

Computed via the log-sum-exp trick with a detached max-shift for numerical stability.

Arguments

• tensor - The tensor.
• dim - The dimension along which log-softmax is computed.

Returns

The output tensor.

fn softmin( tensor: <B as BackendTypes>::FloatTensorPrimitive, dim: usize, ) -> <B as BackendTypes>::FloatTensorPrimitive

Applies the softmin function along the given dimension.

Equivalent to softmax(-tensor, dim).

Arguments

• tensor - The tensor.
• dim - The dimension along which softmin is computed.

Returns

The output tensor.

fn log_sigmoid_backward( x: <B as BackendTypes>::FloatTensorPrimitive, grad: <B as BackendTypes>::FloatTensorPrimitive, ) -> <B as BackendTypes>::FloatTensorPrimitive

Applies the LogSigmoid activation function backward.

Arguments

• x - The input tensor.
• grad - The gradient.

Returns

The output gradient.

Dyn Compatibility

This trait is not dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors