1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
//! Runtime-dispatched SIMD kernels (AVX2 / AVX-512, with a scalar fallback).
//!
//! Covers the two hot loops that dominate CPU time: the meet-solver's
//! candidate-vertex enumeration (`f64`) and the spectral raytracer's
//! plane-intersection and per-channel absorption math (`f32`).
//!
//! # Determinism contract
//!
//! Every kernel is **bit-identical across dispatch levels** (scalar, AVX2,
//! AVX-512) and to the scalar code it replaces, with one exception:
//! [`exp_f32x8`] is a polynomial exponential, bit-identical across its own
//! levels but deliberately **not** identical to `f32::exp` (libm) -- callers
//! switching to it change results by a couple of ULP, once, uniformly.
//! Non-finite inputs are part of that contract: its range clamp follows the
//! `_mm256_min_ps`/`_mm256_max_ps` second-operand rule, so a NaN lane becomes
//! the clamp's upper bound at every level (the scalar lane reproduces this
//! explicitly rather than using `f32::clamp`, which propagates NaN).
//!
//! The `f64`/`f32` geometry kernels replicate the exact operation order of the
//! `glam` scalar code (left-associated dot products, separate multiply and
//! add rather than FMA, per-lane decisions replayed in ascending
//! plane/triple order), so identical operation sequences give identical bits
//! at any vector width. [`exp_f32x8`] is the one FMA user, which is why the
//! dispatch levels also require the `fma` feature.
//!
//! No cross-lane floating-point reductions feed a decision: horizontal steps
//! only *select* existing lane values, ties broken toward the lowest
//! plane/triple index, matching the sequential scans they replace.
pub use exp_f32x8;
pub use exp_lane;
pub use ;
pub use ;
pub use ;
use OnceLock;
/// Which instruction set the kernels dispatch to on this machine.
static LEVEL: = new;
/// Detected dispatch level, cached for the process lifetime.
///
/// `INDICATRIX_SIMD` (read once, at first call) can cap the level below what
/// the CPU supports: `scalar`, `avx2` or `avx512`; never raises it. Since
/// every kernel is bit-identical across levels, capping changes only speed --
/// it exists so PGO training (`scripts/pgo-build.ps1`) can target the scalar
/// kernels on an AVX-capable machine, and so a level can be A/B-timed in place.
const
// Split rather than one function with an internal `#[cfg]`: `is_x86_feature_detected!`
// is a runtime CPUID check, not `const`-compatible, so only the non-x86_64 arm can be `const fn`.
const