Skip to main content

Crate rusttorch_audio

Crate rusttorch_audio 

Source
Expand description

Decode local audio, resample on CPU, and compute differentiable spectral features with LibTorch. Samples use floating-point [channels, frames].

use rusttorch_audio::{Waveform, SpectrogramConfig};
use rusttorch_core::Tensor;
let wave = Waveform::new(Tensor::from_slice(&[0f32; 16]).reshape([1,16]), 8000, 0.)?;
let power = wave.spectrogram(SpectrogramConfig { n_fft: 8, hop_length: 4, power: 2. }, Default::default())?;
assert_eq!(power.size(), [1,5,3]);

§RustTorch Audio

Load waveforms, prepare variable-length audio batches, resample recordings and compute spectral features for speech or sound models. Waveform.samples is a floating-point [channels, frames] tensor accompanied by sample rate, channel layout and a finite time origin.

§Installation

Enable rusttorch = { version = "0.4", features = ["audio"] } for rusttorch::audio, or install rusttorch-audio directly with its download-libtorch feature. The default decoder profile is Rust-native WAV/PCM and FLAC; it does not require FFmpeg. The optional codec dependency exposes the separate FFmpeg adapter when your application explicitly configures native linking.

§Features

The audio facade feature enables WAV/PCM and FLAC decoding, resampling and spectral features. The direct package has no default features. Its optional codec feature re-exports the separate codec package; applications additionally select that package’s system or vcpkg linking feature. download-libtorch obtains the tensor runtime; doc-only builds documentation without native linking and cannot execute tensor operations.

§Native runtime

Tensor operations require LibTorch 2.13.0. Enable download-libtorch, or set LIBTORCH to an extracted distribution and add its library directory to PATH on Windows or the platform’s shared-library search path. All RustTorch packages in one application must share the same runtime. Documentation can be checked with cargo doc -p rusttorch-audio --no-default-features --features doc-only.

§Example: Load and pad audio files

use rusttorch_audio::{AudioFiles, PadAudio};
use rusttorch_data::{DataLoader, ResourceLimits};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let limits = ResourceLimits::default();
    let files = AudioFiles::new(vec!["first.wav".into(), "second.flac".into()], limits)?;
    let mut loader = DataLoader::builder(files)
        .batch_size(8)
        .workers(2)
        .collate(PadAudio { limits })
        .build()?;
    for batch in loader.iter() {
        let batch = batch?;
        // [batch, channels, longest_frames], plus a [batch, longest_frames] mask.
        println!("{:?}, rate={}", batch.samples.size(), batch.sample_rate);
        // Use batch.mask to exclude padding from losses or attention.
    }
    Ok(())
}

A batch requires matching rates, channel layouts, tensor dtypes and devices. Resample or select channels deliberately before batching; the collator never silently changes the recording. lengths records original durations in frames, and padding is zero. Pinning delegates to LibTorch through the common loader.

§Resample and augment a recording

use rusttorch_audio::decode_audio;
use rusttorch_data::ResourceLimits;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let limits = ResourceLimits::default();
    let wave = decode_audio("recording.wav", limits)?
        .resample(16_000, limits)?
        .gain(-3.)?
        .time_shift(160)?;
    println!("{} seconds at {} Hz", wave.duration(), wave.sample_rate);
    Ok(())
}

Rubato resampling uses windowed-sinc interpolation on CPU, removes startup delay and returns ceil(input_frames * new_rate / old_rate) frames. It does not preserve autograd; a tensor requiring gradients must be explicitly detached. Gain, zero-filled time shift, mixing and tensor spectral transforms retain the normal LibTorch graph. mix requires aligned origins, lengths and formats.

For repeatable augmentation inside a loader transform, pass the supplied TaskContext to add_noise(amplitude, context, limits). Noise is uniform in [-amplitude, amplitude] and derives from logical sample identity, seed and epoch. It does not change the global tensor RNG or depend on worker scheduling.

§Compute spectrograms, Mel features and MFCCs

use rusttorch_audio::{decode_audio, SpectrogramConfig, mask_axis};
use rusttorch_data::ResourceLimits;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let limits = ResourceLimits::default();
    let wave = decode_audio("recording.wav", limits)?.resample(16_000, limits)?;
    let stft = SpectrogramConfig { n_fft: 400, hop_length: 160, power: 2. };
    let power = wave.spectrogram(stft, limits)?; // [channels, 201, windows]
    let mel = wave.mel_spectrogram(stft, 40, limits)?;
    let mfcc = wave.mfcc(stft, 40, 13, limits)?;
    let masked = mask_axis(&mel, 1, 3, 4, 0.)?; // Zero four Mel bins.
    assert_eq!(mfcc.size()[1], 13);
    assert_eq!(masked.size(), mel.size());
    println!("spectrogram: {:?}", power.size());
    Ok(())
}

The spectral convention is explicit: periodic Hann window, uncentered complete windows, no signal padding, no FFT normalization, one-sided spectrum. power=1 produces magnitude and power=2 power. Inputs shorter than one window return an error.

Mel filters use HTK frequency spacing from 0 to Nyquist and triangular weights, without area normalization. MFCC uses natural-log Mel energy with floor 1e-10, then orthonormal DCT-II; it does not apply dB or top-dB normalization. Match these choices to how your model was trained. mask_axis copies before masking, so other tensor aliases do not change.

§Formats, safety and evidence

CapabilityVerified scopeEvidence
WAV/FLACLocal bounded files, CPU Float conversiontests/pipelines.rs
Worker loading/paddingEqual-format samples, explicit maskswav_decode_workers_and_padding_preserve_rates_and_lengths
ResamplingCPU sinc interpolation and exact frame countresampling_gain_shift_and_masks_have_explicit_behavior
Spectral featuresValues, shape and gradientsspectral_features_retain_gradients_and_known_sine_peak
Input rejectionBad rate/shape and size ceilingsaudio_boundaries_reject_rate_shape_and_oversize

Limits cover encoded bytes, decoded samples, output tensors and media duration. Decoders preserve errors rather than skipping corrupt packets. Native/library scratch memory is not a process-RSS promise. No audio datasets or codecs are downloaded at runtime.

OperationExecutable evidence
WAV/PCM and FLAC decode to identical samplesflac_decodes_the_same_samples_as_pcm_wav
Worker loading, padding and original lengthswav_decode_workers_and_padding_preserve_rates_and_lengths
Sinc resampling, gain, shifts and maskingresampling_gain_shift_and_masks_have_explicit_behavior
STFT/Mel/MFCC values and input gradientsspectral_values_and_gradients_match_pinned_python
Projected Mel output limitsmel_expansion_checks_output_shape_before_projection

Padded batch limits include waveform bytes, Boolean masks and original lengths, including empty recordings. Record and media-duration limits still apply. Noise generation and resampling check copied host input bytes before allocating.

§Process an FFmpeg audio block

With the direct package’s codec feature, the codec types are available as rusttorch_audio::codec. Select system or vcpkg on rusttorch-codec to create its native decoder. Convert each owned audio block into a waveform and reuse the same preprocessing operations:

fn prepare(block: rusttorch_audio::codec::AudioFrame)
    -> rusttorch_audio::Result<rusttorch_audio::Waveform>
{
    let origin = block.timestamp.map_or(0.0, |time| time.seconds());
    let waveform = rusttorch_audio::Waveform::new(block.samples, block.sample_rate, origin)?;
    waveform.gain(-3.0)
}

The conversion moves the tensor and preserves the source rate and presentation time. A negative time origin is rejected by Waveform; applications retaining negative preroll timestamps must choose an explicit rebasing policy first.

Re-exports§

pub use rusttorch_codec as codec;

Structs§

AudioBatch
Zero-padded equal-rate waveform batch.
AudioFiles
Local audio files decoded independently by loader workers.
PadAudio
Pads waveforms after validating equal rate/layout/dtype/device and byte limits.
SpectrogramConfig
Uncentered STFT configuration, with periodic Hann window and no normalization.
Waveform
Owned waveform metadata and floating-point samples.

Enums§

AudioError
Audio decoding, configuration, resampling or tensor error.
ChannelLayout
Channel ordering carried with decoded samples.

Functions§

decode_audio
Decodes bounded local WAV/FLAC through Symphonia into CPU Float planar audio. Decoder errors are returned, never skipped; sample-rate/layout changes fail.
mask_axis
Masks a contiguous frequency/time axis with a finite constant, without changing the input’s storage. Axis and interval must be in bounds.
mel_filter_bank
Builds Double [n_mels,n_fft/2+1] HTK triangular filters on CPU.

Type Aliases§

Result
Result returned by audio operations.