Skip to main content

Module vexp

Module vexp 

Source
Expand description

The ggml_v_expf polynomial, shared.

attention (softmax) and matmul (GELU/SiLU) each had a private copy of these nine constants. They were byte-identical, and nine duplicated magic numbers in two hot kernels is a drift hazard: a correction applied to one copy and not the other is invisible until two paths disagree about a number.

What is NOT shared is the clamp, and that is deliberate. A softmax argument is always <= 0, so attention clamps below only. A GELU or SiLU argument is unbounded and appears as a DENOMINATOR, so a saturated exponential there divides a numerator with no bound of its own: matmul must select zero above the clamp rather than merely clamp. Each file keeps its own clamp constant next to the code that relies on it.

Source: ggml/src/ggml-cpu/vec.h, ARM optimized-routines expf.

Constants§

EXP_C0
Minimax coefficients for e^b - 1 on [-ln2/2, ln2/2]: 0x1.ffffecp-1, 0x1.fffdb6p-2, 0x1.555e66p-3, 0x1.573e2ep-5, 0x1.0e4020p-7.
EXP_C1
EXP_C2
EXP_C3
EXP_C4
EXP_LN2_HI
High half of ln 2, 0x1.62e4p-1, chosen with trailing zero bits so n * ln2_hi is exact in f32.
EXP_LN2_LO
Low half of ln 2, 0x1.7f7d1cp-20.
EXP_LOG2E
log2(e), 0x1.715476p+0.
EXP_SHIFT
0x1.8p23: adding this to x*log2(e) rounds it to an integer and parks that integer in the mantissa’s low bits.