Skip to main contentModule activations
Source Expand description
- gelu
- GELU 激活函数:f(x) = x * Φ(x) ≈ x * 0.5 * (1 + tanh(√(2/π) * (x + 0.044715 * x³)))
- leaky_relu
- Leaky ReLU 激活函数:f(x) = x if x > 0 else alpha * x
- relu
- ReLU 激活函数:f(x) = max(0, x)
- sigmoid
- Sigmoid 激活函数:f(x) = 1 / (1 + exp(-x))
- softmax
- Softmax 函数(沿指定轴)
- tanh
- Tanh 激活函数:f(x) = tanh(x)