Expand description
The ggml_v_expf polynomial, shared.
attention (softmax) and matmul (GELU/SiLU) each had a private
copy of these nine constants. They were byte-identical, and nine
duplicated magic numbers in two hot kernels is a drift hazard: a
correction applied to one copy and not the other is invisible until
two paths disagree about a number.
What is NOT shared is the clamp, and that is deliberate. A softmax
argument is always <= 0, so attention clamps below only. A GELU
or SiLU argument is unbounded and appears as a DENOMINATOR, so a
saturated exponential there divides a numerator with no bound of its
own: matmul must select zero above the clamp rather than merely
clamp. Each file keeps its own clamp constant next to the code that
relies on it.
Source: ggml/src/ggml-cpu/vec.h, ARM optimized-routines expf.
Constants§
- EXP_C0
- Minimax coefficients for
e^b - 1on[-ln2/2, ln2/2]:0x1.ffffecp-1,0x1.fffdb6p-2,0x1.555e66p-3,0x1.573e2ep-5,0x1.0e4020p-7. - EXP_C1
- EXP_C2
- EXP_C3
- EXP_C4
- EXP_
LN2_ HI - High half of
ln 2,0x1.62e4p-1, chosen with trailing zero bits son * ln2_hiis exact inf32. - EXP_
LN2_ LO - Low half of
ln 2,0x1.7f7d1cp-20. - EXP_
LOG2E log2(e),0x1.715476p+0.- EXP_
SHIFT 0x1.8p23: adding this tox*log2(e)rounds it to an integer and parks that integer in the mantissa’s low bits.