Skip to main content

Crate taconite_reuse

Crate taconite_reuse 

Source
Expand description

NVIDIA RE-USE (nvidia/RE-USE, universal speech enhancement: SEMamba, 30 bidirectional time-frequency Mamba blocks between a dense convolutional encoder and two decoders) on an AMD XDNA NPU: noisy, reverberant, clipped, band-limited or codec-damaged speech at 8 to 48 kHz in, enhanced speech at the same rate out.

The bundle iron/applications/reuse/export_reuse.py writes holds every compiled IRON kernel and the weights (the NPU ones pre-packed); this crate replays the forward the Python app runs (reuse_common.py / reuse_npu.py):

NPUhost (here)
STFT (n_fft 320 at 8 kHz, scaled with the rate; hop 1/8 of it)stft.rs (and its inverse)
encoder (dense block of 4 dilated 3 x 3 convs, a strided conv)every conv as an flm.GEMMthe 1 x 1 input conv, the map layouts, the dilated taps’ sums, InstanceNorm + PReLU
30 x (time Mamba, frequency Mamba), both directionsin_proj, causal conv1d + x_proj, the selective scan, the output projection (the MambaScan operator’s phases)token layouts, both directions summed, LayerNorm, residual
decoders (dense block, two sub-pixel upsampling convs)every conv as an flm.GEMMlayouts, InstanceNorm + PReLU, the 1 x 1 heads, atan2

Reuse::enhance takes any length: up to the bundle’s longest Mamba grid (t_max, ~5 s) in one pass, as the checkpoint’s inference.py; longer audio in 5 s chunks at a 2.5 s hop, overlap-added under a Hann window, as its inference_chunk.py.

Modules§

media
Compressed audio in (feature decode): MP3, AAC in MP4 / M4A, FLAC and WAV through symphonia, and a resampler for rates the bundle has no kernels for.
model
The forward the Python app runs (reuse_common.py), host side here and every convolution, projection and Mamba scan on the NPU:
npu
The NPU side: the bundle’s kernels (kernels naming the same xclbin share its hardware context: one an flm.GEMM shape group and one a MambaScan phase, 7 of NPU2’s 16, shared by every process), flat bf16 device buffers, and the dispatches.
stft
The model’s STFT and its inverse, as torch.stft / torch.istft compute them in the checkpoint’s inference.py: a periodic Hann window of the FFT size, centred frames (the signal reflect-padded by n_fft / 2), one-sided spectra; the magnitude compressed by log1p. Transforms in f64.
wav
WAV files: PCM 16 / 24 / 32-bit and IEEE float 32 in (as torchaudio scales them: int16 / 32768, …), 16-bit PCM or float 32 out.

Structs§

Config
The model’s constants and the bundle’s layout (the manifest’s params).
Reuse
Timing
Wall time per stage, in first-seen order; NPU dispatch time is kept under npu:<kernel>.

Enums§

Error

Constants§

VERSION

Functions§

snr
SNR of x against reference, in dB.