Struct Avx2Math 

pub struct Avx2Math;

Concrete implementation of the SimdMath trait for processors with AVX2 and FMA support.

This is where we “connect the wires”: we connect the abstract mathematical operations of the system to the ultra-fast functions documented above. This struct ensures that NAM-rs takes full advantage of modern hardware to process audio in real time.

Trait Implementations

impl SimdMath for Avx2Math

unsafe fn fused_add_gemm_batch( in_frames: &[f32], weights: &[f32], bias: &[f32], out_frames: &mut [f32], num_frames: usize, do_bias: bool, )

Performs matrix multiplication on a batch of vectors via AVX2. Useful when processing multiple audio frames concurrently to reduce overheads.

unsafe fn fused_gemm_residual_batch( in_frames: &[f32], weights: &[f32], bias: &[f32], residual: &[f32], out_frames: &mut [f32], num_frames: usize, do_bias: bool, )

Performs matrix-vector multiplication also adding the residual connection (skip connection) from the previous layer. Widely used in the WaveNet residual block architecture.

unsafe fn gemv_overwrite( in_frame: &[f32], weights: &[f32], bias: &[f32], out_frame: &mut [f32], do_bias: bool, )

Version that overwrites the output buffer directly with the matrix-vector multiplication result, without accumulating with pre-existing values in the buffer.

unsafe fn gemv_overwrite_bf16( _in_frame: &[u16], _weights: &[u16], _bias: &[f32], _out_frame: &mut [f32], _do_bias: bool, )

Version that overwrites the output buffer accepting input data represented in BF16 (16-bit) and BF16 weights, performing accumulation in f32 to preserve fidelity.

unsafe fn gemv_overwrite_bf16_4gate( _in_frame: &[u16], _weights: &[u16], _bias: &[f32], _out_gates: &mut [f32], _hidden_size: usize, _do_bias: bool, )

Equivalent to gemv_overwrite_4gate but processing input data represented in the BF16 reduced precision format.

unsafe fn store_bf16(ptr: *mut u16, v: Self::V)

Converts a register containing 8 32-bit floats (Self::V) to the compact BF16 (16-bit) format and stores the results in memory.

Bitwise SIMD Magic Details:

To convert f32 to BF16 without spending many CPU cycles on slow mathematical conversions, the technique leverages the structural similarity between IEEE 754 single-precision float and BF16: Both share the same dynamic range (8 exponent bits), but BF16 discards the 16 least significant mantissa bits (truncation/rounding).

const ISA: InstructionSet = InstructionSet::Avx2

ISA that this implementation targets (compile-time constant for monomorphization).

type V = __m256

SIMD register type used (e.g.: __m256 or __m512).

unsafe fn dot_product(a: &[f32], b: &[f32]) -> f32

Computes the dot product between two f32 vectors.

unsafe fn dot_product_4x_interleaved( weights: &[[u16; 4]], state: &[f32], ) -> [f32; 4]

Computes 4 simultaneous BF16 dot products (interleaved) with f32 input.

unsafe fn dot_product_4x_interleaved_dual_frame( weights: &[[u16; 4]], state_f0: &[f32], state_f1: &[f32], ) -> ([f32; 4], [f32; 4])

Computes 4 simultaneous BF16 dot products (interleaved) for 2 parallel frames. Returns (results_f0, results_f1).

unsafe fn dot_product_4x_f32(weights: &[[f32; 4]], state: &[f32]) -> [f32; 4]

Computes 4 simultaneous dot products with native f32 weights.

unsafe fn dot_product_4x_f32_dual( weights: &[[f32; 4]], state_f0: &[f32], state_f1: &[f32], ) -> ([f32; 4], [f32; 4])

Computes 4 simultaneous dot products with native f32 weights for 2 parallel frames. Returns (results_f0, results_f1).

unsafe fn dot_product_8x_f32(weights: &[[f32; 8]], state: &[f32]) -> [f32; 8]

Computes 8 simultaneous dot products with native f32 weights.

unsafe fn dot_product_8x_f32_dual( weights: &[[f32; 8]], state_f0: &[f32], state_f1: &[f32], ) -> ([f32; 8], [f32; 8])

Computes 8 simultaneous dot products with native f32 weights for 2 parallel frames. Returns (results_f0, results_f1).

unsafe fn dot_product_16x_f32(weights: &[[f32; 16]], state: &[f32]) -> [f32; 16]

Computes 16 simultaneous dot products with native f32 weights.

unsafe fn dot_product_16x_f32_dual( weights: &[[f32; 16]], state_f0: &[f32], state_f1: &[f32], ) -> ([f32; 16], [f32; 16])

Computes 16 simultaneous dot products with native f32 weights for 2 parallel frames. Returns (results_f0, results_f1).

unsafe fn dot_product_4x_f32_accumulate( weights: &[[f32; 4]], state: &[f32], init: &[f32; 4], ) -> [f32; 4]

Fused accumulate variant: computes 4 simultaneous dot products with native f32 weights and adds the init accumulator (bias + mixin).

unsafe fn dot_product_4x_f32_dual_accumulate( weights: &[[f32; 4]], state_f0: &[f32], state_f1: &[f32], init_f0: &[f32; 4], init_f1: &[f32; 4], ) -> ([f32; 4], [f32; 4])

Fused accumulate dual-frame variant: computes 4 simultaneous dot products for 2 parallel frames and adds the respective init_f0 / init_f1 accumulators. Returns (results_f0, results_f1).

unsafe fn dot_product_8x_f32_accumulate( weights: &[[f32; 8]], state: &[f32], init: &[f32; 8], ) -> [f32; 8]

Fused accumulate variant: computes 8 simultaneous dot products with native f32 weights and adds the init accumulator.

unsafe fn dot_product_8x_f32_dual_accumulate( weights: &[[f32; 8]], state_f0: &[f32], state_f1: &[f32], init_f0: &[f32; 8], init_f1: &[f32; 8], ) -> ([f32; 8], [f32; 8])

Fused accumulate dual-frame variant: computes 8 simultaneous dot products for 2 parallel frames and adds the respective init_f0 / init_f1 accumulators. Returns (results_f0, results_f1).

unsafe fn dot_product_16x_f32_accumulate( weights: &[[f32; 16]], state: &[f32], init: &[f32; 16], ) -> [f32; 16]

Fused accumulate variant: computes 16 simultaneous dot products with native f32 weights and adds the init accumulator.

unsafe fn dot_product_16x_f32_dual_accumulate( weights: &[[f32; 16]], state_f0: &[f32], state_f1: &[f32], init_f0: &[f32; 16], init_f1: &[f32; 16], ) -> ([f32; 16], [f32; 16])

Fused accumulate dual-frame variant: computes 16 simultaneous dot products for 2 parallel frames and adds the respective init_f0 / init_f1 accumulators. Returns (results_f0, results_f1).

unsafe fn fused_add_gemv( in_frame: &[f32], weights: &[f32], bias: &[f32], out_frame: &mut [f32], do_bias: bool, )

Fused add + GEMV kernel (f32 weights).

unsafe fn fused_gemm_residual_batch_f32( in_frames: &[f32], weights: &[f32], bias: &[f32], residual: &[f32], out_frames: &mut [f32], num_frames: usize, do_bias: bool, )

Fused residual batch GEMM kernel with native f32 weights.

unsafe fn gemv_overwrite_4gate( _in_frame: &[f32], _weights: &[u16], _bias: &[f32], _out_gates: &mut [f32], _hidden_size: usize, _do_bias: bool, )

GEMV with overwrite for 4 simultaneous LSTM gates (f16c-quantized).

unsafe fn gemv_overwrite_batch( in_frames: &[f32], weights: &[f32], bias: &[f32], out_frames: &mut [f32], num_frames: usize, do_bias: bool, )

GEMV overwrite in batch (f32 weights).

unsafe fn gemv_with_bias_f32( in_frames: &[f32], weights: &[f32], bias: &[f32], out_frames: &mut [f32], num_frames: usize, )

GEMV overwrite in batch using native f32 weights (always adds bias).

unsafe fn gemv_no_bias_f32( in_frames: &[f32], weights: &[f32], out_frames: &mut [f32], num_frames: usize, )

GEMV overwrite in batch using native f32 weights (no bias).

unsafe fn batch_norm_process( data: &mut [f32], scale: &[f32], offset: &[f32], n_ch: usize, num_frames: usize, )

Frame-major batch normalization affine transform.

unsafe fn accumulate_head(dest: &mut [f32], src: &[f32])

Accumulates src into dest: dest[i] += src[i].

unsafe fn tanh_and_accumulate_block(head_input: &mut [f32], block: &mut [f32])

Fused Tanh + Head Accumulate.

unsafe fn gated_activation_and_accumulate_block( head_input: &mut [f32], block: &mut [f32], ch: usize, )

Fused Gated Activation + Head Accumulate.

unsafe fn tanh_and_overwrite_block(head_input: &mut [f32], block: &mut [f32])

Fused Tanh + Head Overwrite.

unsafe fn tanh_and_accumulate_with_seed( head_input: &mut [f32], block: &mut [f32], seed: &[f32], )

Fused Seed + Tanh + Head Accumulate.

unsafe fn gated_activation_and_overwrite_block( head_input: &mut [f32], block: &mut [f32], ch: usize, )

Fused Gated Activation + Head Overwrite.

unsafe fn tanh_slice(slice: &mut [f32])

Applies Tanh element-wise to slice.

unsafe fn sigmoid_slice(slice: &mut [f32])

Applies Sigmoid element-wise to slice.

unsafe fn tanh_slice_hf(slice: &mut [f32])

Applies high-fidelity Tanh element-wise to slice.

unsafe fn sigmoid_slice_hf(slice: &mut [f32])

Applies high-fidelity Sigmoid element-wise to slice.

unsafe fn relu_slice(slice: &mut [f32])

Applies ReLU element-wise to slice.

unsafe fn prelu_slice(slice: &mut [f32], slopes: &[f32])

Applies PReLU element-wise: slice[i] = max(0, slice[i]) + slopes[i] * min(0, slice[i]).

unsafe fn softsign_slice(slice: &mut [f32])

Applies Softsign element-wise to slice.

unsafe fn silu_slice(slice: &mut [f32])

Applies SiLU element-wise to slice.

unsafe fn hard_tanh_slice(slice: &mut [f32])

Applies HardTanh element-wise to slice.

unsafe fn hard_swish_slice(slice: &mut [f32])

Applies HardSwish element-wise to slice.

unsafe fn fast_tanh_slice(slice: &mut [f32])

Applies FastTanh (rational approximation) element-wise to slice.

unsafe fn leaky_hard_tanh_slice( slice: &mut [f32], min_val: f32, max_val: f32, min_slope: f32, max_slope: f32, )

Applies LeakyHardTanh element-wise.

unsafe fn activation_tanh_block(buf: &mut [f32])

Tanh activation on a block (shorthand for tanh_slice).

unsafe fn fused_lstm_gates_dyn( gates: &mut [f32], cell_state: &mut [f32], cell_error: &mut [f32], hidden_state: &mut [f32], hidden_size: usize, )

Fused kernel for dynamic LSTM gate processing.

unsafe fn convolve_stereo( coeffs: *const f32, input_l: *const f32, input_r: *const f32, taps: usize, ) -> (f32, f32)

Stereo convolution (used in the resampler).

unsafe fn convolve_stereo_dual( coeffs0: *const f32, coeffs1: *const f32, input_l: *const f32, input_r: *const f32, taps: usize, ) -> ((f32, f32), (f32, f32))

Dual stereo convolution (reuses input loads).

unsafe fn convolve_mono( coeffs: *const f32, input: *const f32, taps: usize, ) -> f32

Mono convolution (used in the resampler).

unsafe fn convolve_mono_dual( coeffs0: *const f32, coeffs1: *const f32, input: *const f32, taps: usize, ) -> (f32, f32)

Dual mono convolution (reuses input loads).

unsafe fn apply_gain_and_detect_clipping_mono( data: &mut [f32], gain: f32, ) -> bool

Applies gain and detects clipping in mono.

unsafe fn apply_gain_and_detect_clipping_stereo( left: &mut [f32], right: &mut [f32], gain: f32, ) -> bool

Applies gain and detects clipping in stereo.

unsafe fn apply_gain_stereo(left: &mut [f32], right: &mut [f32], gain: f32)

Applies constant gain in stereo without clipping detection.

unsafe fn apply_gain(data: &mut [f32], gain: f32)

Applies constant gain to a mono buffer.

unsafe fn apply_ramp(data: &mut [f32], start: f32, step: f32)

Applies a linear gain ramp to a mono buffer.

unsafe fn apply_ramp_stereo( left: &mut [f32], right: &mut [f32], start: f32, step: f32, )

Applies a linear gain ramp in stereo.

unsafe fn apply_gain_then_dither(data: &mut [f32], gain: f32, offset: f32)

Fused gain + dither: data[i] = data[i] * gain + offset in a single pass.

unsafe fn apply_dither_add(data: &mut [f32], offset: f32)

Adds a broadcast constant (dither offset) to every element of a mono buffer.

unsafe fn crossfade_blend_mono(out: &mut [f32], pending: &[f32], t: f32)

Crossfade blend: out[i] = out[i] * (1 - t) + pending[i] * t.

unsafe fn dot_product_bf16(_a: &[u16], _b: &[u16]) -> f32

Computes the dot product between two BF16 vectors.

unsafe fn dot_product_bf16_4x( _w0: &[u16], _w1: &[u16], _w2: &[u16], _w3: &[u16], _in_frame: &[u16], ) -> [f32; 4]

Computes 4 simultaneous BF16 dot products with separate weight vectors.

unsafe fn f32_to_bf16(_src: &[f32], _dest: &mut [u16])

Conversion from F32 to BF16: dest[i] = bf16(src[i]).

unsafe fn horizontal_sum<const N: usize>(ptr: *const f32) -> f32

Horizontal sum of N consecutive f32 values starting at ptr.

unsafe fn compute_energy_stereo(l: &[f32], r: &[f32]) -> f32

Computes max(energy(l), energy(r)) as max mean-square energy.

unsafe fn compute_energy(data: &[f32]) -> f32

Computes the mean-square energy: (1/N) * Σ x_i².

unsafe fn compute_max_diff(a: &[f32], b: &[f32]) -> f32

Computes max(|a[i] - b[i]|).

unsafe fn compute_peak_abs_stereo(left: &[f32], right: &[f32]) -> (f32, f32)

Computes (max(|left|), max(|right|)).

unsafe fn compute_peak_abs_mono(data: &[f32]) -> f32

Computes max(|x[i]|) for a single channel.

unsafe fn complex_mac_overwrite( h_re: &[f32], h_im: &[f32], x_re: &[f32], x_im: &[f32], out_re: &mut [f32], out_im: &mut [f32], )

Complex multiply-accumulate (overwrite): spectral kernel.

unsafe fn complex_mac_accumulate( h_re: &[f32], h_im: &[f32], x_re: &[f32], x_im: &[f32], acc_re: &mut [f32], acc_im: &mut [f32], )

Complex multiply-accumulate (accumulate): spectral kernel.

unsafe fn fft_butterfly_stage( re: *mut f32, im: *mut f32, half: usize, tw_re: *const f32, tw_im: *const f32, group_start: usize, inverse: bool, )

SIMD Radix-2 DIT FFT butterfly stage for one group.

const IS_BF16: bool = false

Indicates whether this implementation uses weights and signals in BF16 format.