Thermite: Melt your CPU

Thermite is a Rust library for portable SIMD programming, providing abstractions over various SIMD instruction sets as well as generic implementations, enabling high-performance vectorized computations across different hardware architectures with ease.

A key aspect of Thermite's design is that it allows you to write a function once (with generics) and evaluate it on any supported backend.

[dependencies]
thermite = "0.2.0"

use thermite::prelude::*;
use thermite::math::TranscendentalMath;

// The logistic sigmoid, written once against trait bounds: no ISA, lane count,
// or element type is named. It compiles for every backend, and the very same
// function also accepts `Dual` (autodiff) or `Compensated` (double-double).
#[thermite::dispatch(V)]
fn sigmoid<V: FloatVector + TranscendentalMath>(x: V) -> V {
    V::ONE / (V::ONE + (-x).exp())
}

let mut data: Vec<f32> = (0..1000).map(|i| i as f32 * 0.01 - 5.0).collect();

let data = data.as_mut_slice(); // dispatch_dyn! takes its parameters by value
thermite::dispatch_dyn!(for<S> |data: &mut [f32]| {
    // `f32xN` is the widest native f32 vector of the ISA selected at runtime,
    // so this is one AVX2 kernel on a modern x86 CPU and one SSE2 kernel on an
    // old one, from a single source.
    let (head, chunks, tail) = data.try_aligned_simd_iter_mut::<f32xN>();

    for v in chunks {
        *v = sigmoid(*v);
    }

    // The ragged ends run through the same `sigmoid` on the 1-lane scalar backend.
    for x in head.iter_mut().chain(tail) {
        *x = sigmoid(Vector::<f32>::splat(*x)).extract::<0>();
    }
});

assert!((data[500] - 0.5).abs() < 1e-6); // sigmoid(0) == 0.5

The Rules

Read these before writing anything. They are not style advice, and breaking rule 1 in particular is worse than not using SIMD at all.

0. Do not touch the Register layer unless you know exactly what you're doing. Stick with the *Vector traits.
1. #[thermite::dispatch] and #[inline(always)] are MANDATORY, or else your code will be abysmally slow.
2. Prefer trait bounds over concrete vector types. Commit to a fixed lane count when the data structure is genuinely defined by it, not out of convenience.
3. Thermite isn't magic. You must consider what you're doing before you expect it to be fast.
4. Look for any built-in methods before re-implementing things yourself. All of Thermite's standard library is highly optimized.
5. Avoid scalar work whenever possible.
6. Doing nothing is better than doing something clever.

Why rule 1 is not a suggestion

rustc will not inline a #[target_feature] function into a caller that lacks those features, and every intrinsic is one. A generic SIMD body with no #[thermite::dispatch] above it compiles featureless: every operation becomes an out-of-line call, with no scheduling or register allocation across them. It still compiles. It is still correct. However, it is catastrophically slow.

Two attributes avoid it. Put #[thermite::dispatch] on the outermost SIMD entry point, and #[inline(always)] on the helpers called beneath it. The guide covers the details, and this is the first thing to check when a kernel underperforms.

What's in it

The trait ladder runs GenericVector to NumericVector to FloatVector, with the math traits on top. A kernel is bounded on the weakest one it needs.

Beyond arithmetic and the usual transcendentals, the vector API covers stream compaction (compress / expand), lane prefix scans, duplicate-lane conflict detection, gather and scatter, the full N-by-N cast matrix across element types, interleaving at group granularity, and packed fp16 / bf16 / fp8 storage.

The math library is policy-configurable. Every function has a _p form taking a P: Policy, so the same exp call can be tuned from UltraPerformance to Reference at the call site instead of through a second set of function names.

Backends

Which backends are compiled in the first place is covered under Reaching the other backends below.

Feature flags

Default: document_registers, bitvec, avx2-f16c, avx2-pclmul.

Numerics, and what they change about results

These change the numbers coming out, not just the speed.

Combinations resolve rather than conflict, which matters because Cargo unifies features across an entire dependency graph and an unrelated crate can switch one on. strict_ieee754 wins over algebraic-scalar, and preserve_denormals wins over ignore_denormals. The resolved answer is readable at compile time from the thermite::features module (STRICT_IEEE754, ALGEBRAIC_SCALAR, ...), so downstream code can gate on it or reject it with a const assertion instead of re-deriving the precedence itself.

Codegen

API surface

std (off by default, this crate is no_std) enables formatted panic messages and num-traits/std. bitvec (default) adds the bitvec mask integration. partial-ord adds PartialOrd for Vector, which only holds when every lane shares the order. document_registers (default) is documentation only.

Reaching the other backends

The x86 backends need no feature at all. x86_v1, x86_v2 and x86_v3 are compiled unconditionally on x86 and x86_64, and dispatch_dyn! picks between them at runtime, so one binary carries all three.

AArch64 NEON is the same story with no flag: AdvSIMD is mandatory in the architecture, so the backend is gated on target_arch = "aarch64" and is always compiled there. There is deliberately no neon feature, and passing one is an error. 32-bit ARM is not supported, since its NEON intrinsics are still unstable and ARMv7 NEON has no f64 lanes.

WebAssembly is opt-in, because SIMD128 is a proposal an engine may not have enabled:

thermite = { version = "0.2.0", default-features = false, features = ["wasm"] }

wasm32 works on stable. wasm64 additionally needs nightly.

AVX-512 has no backend yet. The avx512-tier1 through avx512-tier4 features are reserved names that select nothing today, kept so the tier names stay stable when the backend lands. Enabling them changes no codegen, and AVX-512 hardware runs the AVX2 backend in the meantime.

SPIR-V is unfinished, and enabling the spirv feature on a released version is a hard compile error. Work on it continues and the code stays in the repository. If you want to build it regardless, take a git dependency and set RUSTFLAGS='--cfg thermite_unstable_spirv'.

Companion crates

The same generic functions run unmodified on the composite types, which is the point of writing them against traits in the first place.

• thermite-special for the error function, gamma, elliptic integrals and activations
• thermite-dual for forward-mode automatic differentiation
• thermite-compensated for double-double precision
• thermite-complex for complex numbers
• thermite-sort, thermite-sdf, thermite-geometry for sorting, signed distance fields, and SoA geometric primitives

More

There is a full guide covering the trait hierarchy, slice iteration, masks, the policy system and performance work. It ships with the crate and renders as the thermite::guide module on docs.rs.

Stable Rust, MSRV 1.95, edition 2024. no_std by default.

License

MIT or Apache-2.0, at your option.