Skip to main content

Module gelu

Module gelu 

Source
Expand description

GELU activation (pytorch_tanh variant) GPU dispatch.

Computes: 0.5 * x * (1 + tanh(sqrt(2/pi) * (x + 0.044715 * x^3)))

This is the exact variant used by Gemma 4. It is not the erf-based GELU approximation.

Statics§

GELU_SHADER_SOURCE
MSL source for the GELU kernels (embedded at compile time).

Functions§

dispatch_gelu
Dispatch a GELU activation on the GPU.
register
Register GELU shader sources with the given kernel registry.