Expand description
Tensor-level activations on crate::fastmath — the drop-in replacements
for candle’s, which compute a scalar libm call per element.
§What this is worth
Measured in ffai-argus on (1, 1024, 3072), the shape a SigLIP MLP
actually runs: candle’s .gelu() took 44.01 ms, this shape of kernel
1.22 ms — 32x, with the caption it feeds byte-identical to the reference
implementation’s. candle’s activations are elementwise ops in a backend that
uses rayon for conv2d and nothing else, evaluating tanhf/erf per
element on one core.
§Two things that are NOT interchangeable
gelu_erf is 0.5x(1 + erf(x/sqrt 2)); gelu_tanh is the tanh
approximation of it. They differ by up to ~1e-3 — far above the
tolerance these engines gate at — so a site calling one must be given that
one. Six of the seventeen activation sites in this workspace are gelu_erf
and three are gelu_tanh; swapping them silently would be a quality
regression that no test here would catch, because both are “a GELU”.
§Delivery
Each op is a candle_core::CustomOp1, which hands the kernel candle’s own
CpuStorage. That matters more than the arithmetic: routing a tensor
through to_vec1() and Tensor::from_vec is a fixed per-call tax, and in
the Argus campaign more than half the total win came from removing the glue
rather than from the polynomial (3.03 ms -> 2.30 -> 1.08 across three
delivery fixes with no change to the inner loop).