ActivationOps

burn_tensor::ops

Trait ActivationOps 

Source 
pub trait ActivationOps<B: Backend> {
    // Provided methods
    fn leaky_relu(
        tensor: FloatTensor<B>,
        negative_slope: FloatElem<B>,
    ) -> FloatTensor<B> { ... }
    fn relu(tensor: FloatTensor<B>) -> FloatTensor<B> { ... }
    fn relu_backward(
        output: FloatTensor<B>,
        grad: FloatTensor<B>,
    ) -> FloatTensor<B> { ... }
    fn gelu(tensor: FloatTensor<B>) -> FloatTensor<B> { ... }
    fn prelu(tensor: FloatTensor<B>, alpha: FloatTensor<B>) -> FloatTensor<B> { ... }
    fn gelu_backward(x: FloatTensor<B>, grad: FloatTensor<B>) -> FloatTensor<B> { ... }
    fn sigmoid(tensor: FloatTensor<B>) -> FloatTensor<B> { ... }
    fn sigmoid_backward(
        output: FloatTensor<B>,
        grad: FloatTensor<B>,
    ) -> FloatTensor<B> { ... }
    fn hard_sigmoid(
        tensor: FloatTensor<B>,
        alpha: FloatElem<B>,
        beta: FloatElem<B>,
    ) -> FloatTensor<B> { ... }
    fn log_sigmoid(tensor: FloatTensor<B>) -> FloatTensor<B> { ... }
    fn log_sigmoid_backward(
        x: FloatTensor<B>,
        grad: FloatTensor<B>,
    ) -> FloatTensor<B> { ... }
}
Expand description

Activation function operations.

This trait let backend implementations override activation functions for better performance.

Provided Methods§

Source

fn leaky_relu( tensor: FloatTensor<B>, negative_slope: FloatElem<B>, ) -> FloatTensor<B>

Applies the LeakyReLU activation function.

§Arguments
  • tensor - The tensor.
  • negative_slope - The negative_slope value that values smaller than 0 are multiplied with.
§Returns

The output tensor.

Source

fn relu(tensor: FloatTensor<B>) -> FloatTensor<B>

Applies the ReLU activation function.

§Arguments
  • tensor - The tensor.
§Returns

The output tensor.

Source

fn relu_backward(output: FloatTensor<B>, grad: FloatTensor<B>) -> FloatTensor<B>

Applies the ReLU activation function backward.

§Arguments
  • output - The output tensor.
§Returns

The gradient.

Source

fn gelu(tensor: FloatTensor<B>) -> FloatTensor<B>

Applies the Gelu activation function.

§Arguments
  • tensor - The tensor.
§Returns

The output tensor.

Source

fn prelu(tensor: FloatTensor<B>, alpha: FloatTensor<B>) -> FloatTensor<B>

Applies the PReLu activation function.

§Arguments
  • tensor - The input tensor
  • alpha - The weight tensor
Source

fn gelu_backward(x: FloatTensor<B>, grad: FloatTensor<B>) -> FloatTensor<B>

Applies the Gelu activation function backward.

§Arguments
  • x - The tensor.
  • grad - The gradient.
§Returns

The output tensor.

Source

fn sigmoid(tensor: FloatTensor<B>) -> FloatTensor<B>

Applies the Sigmoid activation function.

§Arguments
  • tensor - The tensor.
§Returns

The output tensor.

Source

fn sigmoid_backward( output: FloatTensor<B>, grad: FloatTensor<B>, ) -> FloatTensor<B>

Applies the Sigmoid activation function backward.

§Arguments
  • output - The output tensor of the sigmoid function.
  • grad - The gradient.
§Returns

The output tensor.

Source

fn hard_sigmoid( tensor: FloatTensor<B>, alpha: FloatElem<B>, beta: FloatElem<B>, ) -> FloatTensor<B>

Applies the hard Sigmoid activation function.

§Arguments
  • tensor - The tensor.
  • alpha - The alpha value that the tensor is multiplied with.
  • beta - The beta value that is added to the tensor
§Returns

The output tensor.

Source

fn log_sigmoid(tensor: FloatTensor<B>) -> FloatTensor<B>

Applies the LogSigmoid activation function.

§Arguments
  • tensor - The tensor.
§Returns

The output tensor.

Source

fn log_sigmoid_backward( x: FloatTensor<B>, grad: FloatTensor<B>, ) -> FloatTensor<B>

Applies the LogSigmoid activation function backward.

§Arguments
  • x - The input tensor.
  • grad - The gradient.
§Returns

The output gradient.

Dyn Compatibility§

This trait is not dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety", so this trait is not object safe.

Implementors§