Skip to main content

geglu

Function geglu 

Source
pub fn geglu(gate: &[f32], up: &[f32]) -> Result<Vec<f32>, EngineError>
Expand description

GeGLU: gelu(gate) * up (Gemma gelu_pytorch_tanh approximates with tanh form).