Function dfdx::tensor_ops::gelu

source · 
pub fn gelu<S: Shape, E: Dtype, D: UnaryKernel<GeLUKernelOp, E>, T: Tape<E, D>>(
    t: Tensor<S, E, D, T>
) -> Tensor<S, E, D, T>
Expand description

Gaussian Linear Unit (GeLU). 0.5 * x * (1 + tanh(sqrt(2 / pi) * (x + 0.044715 * x^3)))

Examples:

let t = dev.tensor([-1.0, 0.0, 1.0, 2.0]);
let r = t.gelu();