Crate fearless_simd

Expand description

fearless_simd takes unsafe out of SIMD.

No matter what level of abstraction you’re after, be it autovectorization and multiversioning, or portable SIMD, or safe access to raw intrinsics and nothing more, fearless_simd has you covered!

Zero dependencies, from-scratch build time under 1 second, safe public APIs, and very little unsafe under the hood.

§Automatic vectorization

Put the code to vectorize in an #[inline(always)] function generic over Simd.

This will generate several implementations for different SIMD levels and select the best one at runtime:

use fearless_simd::{dispatch, Level, Simd};

#[inline(always)]
fn double_u32s<S: Simd>(_: S, values: &mut [u32]) {
    for value in values {
        *value = *value * 2;
    }
}

let mut values = [1, 2, 3, 4, 5];
let level = Level::new(); // Detect SIMD available on the CPU. Expensive, so do it once.
dispatch!(level, simd => double_u32s(simd, &mut values));
assert_eq!(values, [2, 4, 6, 8, 10]);

§Portable SIMD

Use the vector types for explicit lane-wise operations while staying generic over the SIMD level:

use fearless_simd::{dispatch, prelude::*, Level};

#[inline(always)]
fn double_u32s<S: Simd>(simd: S, values: &mut [u32]) {
    let mut chunks = values.chunks_exact_mut(S::u32s::N); // the CPU's native SIMD width
    for chunk in &mut chunks {
        let v = S::u32s::from_slice(simd, chunk);
        (v * 2).store_slice(chunk);
    }
    for value in chunks.into_remainder() {
        *value = *value * 2;
    }
}

let mut values = [1, 2, 3, 4, 5];
let level = Level::new(); // Detect SIMD available on the CPU. Expensive, so do it once.
dispatch!(level, simd => double_u32s(simd, &mut values));
assert_eq!(values, [2, 4, 6, 8, 10]);

You can also use fixed-size types such as u32x8 instead of using the hardware’s native SIMD width.

§Explicit intrinsics

If you need access to raw intrinsics, kernel! creates a function where they can be called safely:

use fearless_simd::{prelude::*, Level, u32x4};

fearless_simd::kernel!(
    fn double_u32s_neon(neon: Neon, values: &mut [u32]) {
        use core::arch::aarch64::*;

        let mut chunks = values.chunks_exact_mut(4);
        for chunk in &mut chunks {
            let v: uint32x4_t = u32x4::from_slice(neon, chunk).into(); // safe load
            let doubled = vmulq_u32(v, vdupq_n_u32(2)); // safe access to a NEON intrinsic
            let doubled: u32x4<_> = doubled.simd_into(neon);
            doubled.store_slice(chunk);
        }
        for value in chunks.into_remainder() {
            *value = *value * 2;
        }
    }
);

#[cfg(target_arch = "aarch64")]
{
    let level = Level::new(); // Detect SIMD available on the CPU. Expensive, so do it once.
    if let Some(neon) = level.as_neon() {
        let mut values = [1, 2, 3, 4, 5];
        double_u32s_neon(neon, &mut values);
        assert_eq!(values, [2, 4, 6, 8, 10]);
    }
}

You can also mix and match intrinsics with the other approaches, using high-level code most of the time and dropping down to hardware-specific intrinsics only when necessary.

§Inlining

Fearless SIMD relies heavily on Rust’s inlining support to create functions which have the given target features enabled.

As a rule of thumb:

All SIMD functions need #[inline(always)].
Use dispatch when calling SIMD code from non-SIMD code.
Use vectorize() when calling SIMD from SIMD if you don’t want to force inlining.

The article describing the design covers why this is the case. There’s also Q&A on Zulip.

§Instruction set support

x86/x86-64: v2 (SSE4.2), v3 (AVX2), Ice Lake (AVX-512, avoiding early slow implementations)
Aarch64: Baseline NEON
WebAssembly: 128-bit packed SIMD, relaxed SIMD

A scalar fallback is also provided for platforms, so your code still works even if SIMD is not available.

§WebAssembly

WASM SIMD doesn’t have feature detection, and so you need to compile two versions of your bundle for WASM, one with SIMD and one without, then select the appropriate one for your user’s browser. This can be done via the wasm-feature-detect library.

You can compile WebAssembly with the SIMD128 feature enabled via the RUSTFLAGS environment variable (RUSTFLAGS="-Ctarget-feature=+simd128"), or by adding the compiler flags in your Cargo config.toml:

[target.'cfg(target_arch = "wasm32")']
rustflags = ["-Ctarget-feature=+simd128"]
rustdocflags = ["-Ctarget-feature=+simd128"]

If you want to compile both SIMD and non-SIMD versions of your WebAssembly library, your best option right now is to create a shell script that builds it once with the RUSTFLAGS specified, and once without. Cargo currently does not allow specifying compiler flags per-profile.

§Relaxed SIMD

Fearless SIMD can make use of the relaxed SIMD WebAssembly instructions, if the requisite target feature is enabled. These instructions can return implementation-dependent results depending on what is fastest on the underlying hardware. They are only used for operations where we already give hardware-dependent results.

At the time of writing, relaxed SIMD is only supported in Chrome. To make use of it, you’ll need to build two versions of your library, one with relaxed SIMD enabled (RUSTFLAGS="-Ctarget-feature=+simd128,+relaxed-simd") and one with it disabled, and then feature-detect at runtime.

§Multiversioning on x86

x86 CPUs are not guaranteed to have any SIMD particular instruction set, so fearless_simd compiles a version of each function generic over Simd for each instruction set, and dispatch selects the best one at runtime.

This is necessary to take advantage of SIMD, but results in an increased binary size on x86. If binary size is a concern, the increase can be partially mitigated by setting codegen-units=1 or lto=true in your Cargo.toml, at the cost of longer build times.

As a last resort, you can turn off multiversioning for specific SIMD instruction sets by passing --cfg disable_dispatch_sse4_2, --cfg disable_dispatch_avx2, or --cfg disable_dispatch_avx512 in RUSTFLAGS. These configuration flags only control automatic multiversioning. Disabling one does not remove its token type, its Simd implementation, or explicit kernel support; for example, an Avx2 token can still be used to call an AVX2 kernel when the CPU supports it.

Note that later extensions can be beneficial even if you are only using 128-bit vectors: AVX2 and AVX-512 provide more efficient instructions for some operations, and AVX-512 also more than doubles the number of vector registers of all sizes.

You can also disable certain instruction sets for select functions without disabling them globally.

§Feature Flags

The following crate feature flags are available:

std (enabled by default): Get floating point functions from the standard library (likely using your target’s libc). Also allows using Level::new on all platforms, to detect which target features are enabled.
libm: Use floating point implementations from libm. Useful for #[no_std].
force_support_fallback: Force scalar fallback, to be supported, even if your compilation target has a better baseline.

At least one of std and libm is required; std overrides libm.

§Credits

This crate was inspired by pulp, std::simd, among others in the Rust ecosystem, though makes many decisions differently. It benefited from conversations with Luca Versari, though he is not responsible for any of the mistakes or bad decisions.

Modules§

prelude: This prelude module re-exports every SIMD trait defined in this library. It’s useful for accessing trait methods.
x86x86 or x86-64: Implementations of Simd on x86 architectures (both 32 and 64 bit).

Macros§

dispatch: Access the applicable Simd for a given level, and perform an operation using it.
kernel: Creates a context where you can safely call intrinsics available at the SIMD level named by the function’s first argument.

Structs§

Avx2x86 or x86-64: A token for AVX2 intrinsics on x86 and x86_64, representing the x86-64-v3 level.
Avx512x86 or x86-64: A token for AVX-512 intrinsics on x86 and x86_64, representing an Ice Lake feature level.
Fallback: A token for scalar fallback SIMD, representing the “fallback” level.
Sse4_2x86 or x86-64: A token for SSE4.2 intrinsics on x86 and x86_64, representing the x86-64-v2 level.
f32x4: A SIMD vector of 4 f32 elements.
f32x8: A SIMD vector of 8 f32 elements.
f32x16: A SIMD vector of 16 f32 elements.
f64x2: A SIMD vector of 2 f64 elements.
f64x4: A SIMD vector of 4 f64 elements.
f64x8: A SIMD vector of 8 f64 elements.
i8x16: A SIMD vector of 16 i8 elements.
i8x32: A SIMD vector of 32 i8 elements.
i8x64: A SIMD vector of 64 i8 elements.
i16x8: A SIMD vector of 8 i16 elements.
i16x16: A SIMD vector of 16 i16 elements.
i16x32: A SIMD vector of 32 i16 elements.
i32x4: A SIMD vector of 4 i32 elements.
i32x8: A SIMD vector of 8 i32 elements.
i32x16: A SIMD vector of 16 i32 elements.
mask8x16: A SIMD mask of 16 logical lanes corresponding to 8-bit vector elements.
mask8x32: A SIMD mask of 32 logical lanes corresponding to 8-bit vector elements.
mask8x64: A SIMD mask of 64 logical lanes corresponding to 8-bit vector elements.
mask16x8: A SIMD mask of 8 logical lanes corresponding to 16-bit vector elements.
mask16x16: A SIMD mask of 16 logical lanes corresponding to 16-bit vector elements.
mask16x32: A SIMD mask of 32 logical lanes corresponding to 16-bit vector elements.
mask32x4: A SIMD mask of 4 logical lanes corresponding to 32-bit vector elements.
mask32x8: A SIMD mask of 8 logical lanes corresponding to 32-bit vector elements.
mask32x16: A SIMD mask of 16 logical lanes corresponding to 32-bit vector elements.
mask64x2: A SIMD mask of 2 logical lanes corresponding to 64-bit vector elements.
mask64x4: A SIMD mask of 4 logical lanes corresponding to 64-bit vector elements.
mask64x8: A SIMD mask of 8 logical lanes corresponding to 64-bit vector elements.
u8x16: A SIMD vector of 16 u8 elements.
u8x32: A SIMD vector of 32 u8 elements.
u8x64: A SIMD vector of 64 u8 elements.
u16x8: A SIMD vector of 8 u16 elements.
u16x16: A SIMD vector of 16 u16 elements.
u16x32: A SIMD vector of 32 u16 elements.
u32x4: A SIMD vector of 4 u32 elements.
u32x8: A SIMD vector of 8 u32 elements.
u32x16: A SIMD vector of 16 u32 elements.

Enums§

Level: The level enum with the specific SIMD capabilities available.

Traits§

Bytes: Conversion of SIMD types to and from raw bytes.
Select: Element-wise selection between two SIMD vectors using self.
Simd: The main SIMD trait, implemented by all SIMD token types.
SimdBase: Base functionality implemented by all SIMD vectors.
SimdCombine: Concatenation of two SIMD vectors.
SimdCvtFloat: Construction of floating point vectors from integers
SimdCvtTruncate: Construction of integer vectors from floats by truncation
SimdElement: Types that can be used as elements in SIMD vectors.
SimdFloat: Functionality implemented by floating-point SIMD vectors.
SimdFrom: Value conversion, adding a SIMD blessing.
SimdInt: Functionality implemented by (signed and unsigned) integer SIMD vectors.
SimdInto: Value conversion, adding a SIMD blessing.
SimdMask: Functionality implemented by SIMD masks.
SimdSplit: Splitting of one SIMD vector into two.
WithSimd