1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
//! SIMD abstraction layer and fast math functions.
//!
//! All numerical computation in this module is written generically over the
//! [`crate::simd::generic::SimdFloat`] trait. The default
//! [`crate::simd::generic::ScalarFloat`] backend (WIDTH=1) works everywhere;
//! platform backends are activated by [`crate::sim`]'s compile-time-selected
//! opt-in features, or unconditionally by [`crate::simd`]'s own
//! runtime-detected `simd` feature (see
//! [`crate::simd::generic::dispatch::detect_backend`]):
//!
//! | Feature | Type | WIDTH | Platform |
//! |---------------|------------|-------|-----------|
//! | (none) | `ScalarFloat` | 1 | all |
//! | `simd` | `F32x4Sse` | 4 | x86_64 (baseline floor)|
//! | `sim-avx2` | `F32x8` | 8 | x86_64 |
//! | `sim-avx512` | `F32x16` | 16 | x86_64 |
//! | `sim-neon` | `F32x4` | 4 | aarch64 |
//!
//! # Fast Math
//!
//! The following functions are provided with < 1e-5 relative error:
//!
//! - [`crate::simd::generic::simd_exp`] -- exponential via range reduction + degree-5 polynomial
//! - [`crate::simd::generic::simd_log`] -- natural log via atanh series expansion
//! - [`crate::simd::generic::simd_log1pexp`] -- numerically stable softplus
//! - [`crate::simd::generic::simd_sigmoid`] -- logistic sigmoid without overflow
//! - [`crate::simd::generic::simd_recip`], [`crate::simd::generic::simd_rsqrt`] -- Newton-refined reciprocal/inverse sqrt
// `exp`'s Cody-Waite range-reduction constants are intentionally given to
// more decimal digits than `f32` can represent exactly (for readability
// against reference values), hence this allowance for the whole module.
/// SIMD exponential (`simd_exp`).
/// SIMD natural log and softplus (`simd_log`, `simd_log1pexp`).
/// SIMD reciprocal and inverse square root (`simd_recip`, `simd_rsqrt`).
/// The always-available `WIDTH=1` fallback backend.
/// SIMD logistic sigmoid (`simd_sigmoid`).
/// The [`crate::simd::generic::SimdFloat`] abstraction trait itself.
/// AVX2 `F32x8` backend, `x86_64` only. Compiled whenever either
/// [`crate::sim`]'s compile-time-selected `sim-avx2` backend or
/// [`crate::simd`]'s runtime-dispatched `simd` feature needs it - the two
/// layers share this same type rather than each hand-rolling their own.
/// AVX-512 `F32x16` backend, `x86_64` only. Compiled whenever either
/// [`crate::sim`]'s compile-time-selected `sim-avx512` backend or
/// [`crate::simd`]'s runtime-dispatched `simd` feature needs it, mirroring
/// [`crate::simd::generic::avx2`]'s dual use.
/// ARM NEON `F32x4` backend, `aarch64` only. Compiled whenever either
/// [`crate::sim`]'s compile-time-selected `sim-neon` backend or
/// [`crate::simd`]'s runtime-dispatched `simd` feature needs it, mirroring
/// [`crate::simd::generic::avx2`]'s dual use.
/// Baseline SSE2 `F32x4Sse` backend, `x86_64` only - see
/// [`crate::simd::generic::sse2`]'s module docs for why it needs no opt-in
/// feature of its own.
/// Width-agnostic `dot`/`mul_elementwise`/`mix_scalar` shared by every
/// [`crate::simd::generic::SimdFloat`] backend - see
/// [`crate::simd::generic::ops`]'s module docs.
/// Runtime backend selection ([`crate::simd::generic::dispatch::detect_backend`]).
pub use SimdBackend;
pub use ScalarFloat;
pub use SimdFloat;
pub use simd_exp;
pub use ;
pub use ;
pub use simd_sigmoid;
/// Select the best available SIMD backend at compile time.
///
/// This is [`crate::sim`]'s own compile-time choice (which `sim-*` feature
/// was enabled), distinct from
/// [`crate::simd::generic::dispatch::detect_backend`]'s runtime CPU check -
/// see that function's docs for why the two differ. Returns a string
/// identifying the active backend, from the same vocabulary as
/// [`SimdBackend::as_str`].