Crate spectrograms

Source

Expand description

§Spectrograms - FFT-Based Computations

High-performance FFT-based computations for audio and image processing.

§Overview

This library provides:

1D FFTs: For time-series and audio signals
2D FFTs: For images and spatial data
Spectrograms: Time-frequency representations (STFT, Mel, ERB, CQT)
Image operations: Convolution, filtering, edge detection
Two backends: RealFFT (pure Rust) or FFTW (fastest)
Plan-based API: Reusable plans for batch processing
Precision-generic: Works in f32 or f64 (defaults to f64) via the Sample trait

§Domain Organization

The library is organized by application domain:

audio - Audio processing (spectrograms, MFCC, chroma, pitch analysis)
image - Image processing (convolution, filtering, frequency analysis)
fft - Core FFT operations (1D and 2D transforms)

All functionality is also exported at the crate root for convenience.

§Audio Processing

Compute various types of spectrograms:

Linear-frequency spectrograms
Mel-frequency spectrograms
ERB spectrograms
Logarithmic-frequency spectrograms
CQT (Constant-Q Transform)

With multiple amplitude scales:

Power (|X|²)
Magnitude (|X|)
Decibels (10·log₁₀(power))

§Image Processing

Frequency-domain operations for images:

2D FFT and inverse FFT
Convolution via FFT (faster for large kernels)
Spatial filtering (low-pass, high-pass, band-pass)
Edge detection
Sharpening and blurring

§Features

Two FFT backends: RealFFT (default, pure Rust) or FFTW (fastest performance)
Plan-based computation: Reuse FFT plans for efficient batch processing
Comprehensive window functions: Hanning, Hamming, Blackman, Kaiser, Gaussian, etc.
Type-safe API: Compile-time guarantees for spectrogram types

§Quick Start

§Audio: Compute a Mel Spectrogram

use spectrograms::*;
use std::f64::consts::PI;
use non_empty_slice::NonEmptyVec;

// Generate a sine wave at 440 Hz
let sample_rate = 16000.0;
let samples_vec: Vec<f64> = (0..16000)
    .map(|i| (2.0 * PI * 440.0 * i as f64 / sample_rate).sin())
    .collect();
let samples = NonEmptyVec::new(samples_vec).unwrap();

// Set up parameters
let stft = StftParams::new(nzu!(512), nzu!(256), WindowType::Hanning, true)?;
let params = SpectrogramParams::new(stft, sample_rate)?;
let mel = MelParams::new(nzu!(80), 0.0, 8000.0)?;

// Compute Mel spectrogram
let spec = MelPowerSpectrogram::compute(samples.as_ref(), &params, &mel, None)?;
println!("Computed {} bins x {} frames", spec.n_bins(), spec.n_frames());

§Image: Apply Gaussian Blur via FFT

use spectrograms::image_ops::*;
use spectrograms::nzu;
use ndarray::Array2;

// Create a 256x256 image
let image = Array2::<f64>::from_shape_fn((256, 256), |(i, j)| {
    ((i as f64 - 128.0).powi(2) + (j as f64 - 128.0).powi(2)).sqrt()
});

// Apply Gaussian blur
let kernel = gaussian_kernel_2d(nzu!(9), 2.0)?;
let blurred = convolve_fft(&image.view(), &kernel.view())?;

§General: 2D FFT

use spectrograms::fft2d::*;
use ndarray::Array2;

let data = Array2::<f64>::zeros((128, 128));
let spectrum = fft2d(&data.view())?;
let power = power_spectrum_2d(&data.view())?;

§Feature Flags

The library requires exactly one FFT backend:

realfft (default): Pure-Rust FFT implementation, no system dependencies
fftw: Uses FFTW C library for fastest performance (requires system install)

§Numeric Precision (`f32` / `f64`)

All core computations are generic over the floating-point scalar type via the sealed Sample trait (implemented for f32 and f64). The scalar type defaults to f64, so existing code is unchanged; pass f32 inputs (or annotate the type) to compute in single precision — useful for memory-bound workloads and ML pipelines that train in f32:

use spectrograms::*;
use non_empty_slice::NonEmptyVec;
let sig = NonEmptyVec::new(vec![0.0_f32; 1024]).unwrap();
// `T = f32` is inferred from the input type.
let spec = stft(&sig, nzu!(256), nzu!(128), WindowType::Hanning, true)?;

Coverage spans STFT, Mel/ERB/CQT, MFCC, chroma, MDCT, convolution, minimum-phase, binaural, and 2D FFT / image operations. The f64-input convenience constructors (chromagram, mfcc, gaussian_kernel_2d) return f64; reach single precision via their input-generic variants (chromagram_from_spectrogram, mfcc_from_log_mel) or the primitives.

§Examples

§Mel Spectrogram

use spectrograms::*;
use non_empty_slice::non_empty_vec;

let samples = non_empty_vec![0.0; nzu!(16000)];

let stft = StftParams::new(nzu!(512), nzu!(256), WindowType::Hanning, true)?;
let params = SpectrogramParams::new(stft, 16000.0)?;
let mel = MelParams::new(nzu!(80), 0.0, 8000.0)?;
let db = LogParams::new(-80.0)?;

let spec = MelDbSpectrogram::compute(samples.as_ref(), &params, &mel, Some(&db))?;

§MDCT (Modified Discrete Cosine Transform)

use spectrograms::*;
use non_empty_slice::NonEmptyVec;

let samples: Vec<f64> = (0..4096).map(|i| (i as f64 * 0.01).sin()).collect();
let samples = NonEmptyVec::new(samples).unwrap();

// Sine window gives perfect reconstruction at 50% hop
let params = MdctParams::sine_window(nzu!(512))?;

let coefficients = mdct(samples.as_non_empty_slice(), &params)?;
let reconstructed = imdct(&coefficients, &params, Some(samples.len().get()))?;

§Efficient Batch Processing

use spectrograms::*;
use non_empty_slice::non_empty_vec;

let signals = vec![non_empty_vec![0.0; nzu!(16000)], non_empty_vec![0.0; nzu!(16000)]];

let stft = StftParams::new(nzu!(512), nzu!(256), WindowType::Hanning, true)?;
let params = SpectrogramParams::new(stft, 16000.0)?;

// Create plan once, reuse for all signals
let planner = SpectrogramPlanner::new();
let mut plan = planner.linear_plan::<Power, _>(&params, None)?;

for signal in &signals {
    let spec = plan.compute(&signal)?;
    // Process spec...
}

Re-exports§

pub use source::ChromaSource;
pub use source::CqtSource;
pub use source::GammatoneSource;
pub use source::MfccSource;
pub use source::SpectrogramSource;
pub use fft2d::*;
pub use image_ops::*;

Modules§

audio: Audio processing utilities (spectrograms, MFCC, chroma, etc.)
binaural: Binaural audio analysis spectrograms.
fft: Core FFT operations (1D and 2D)
fft2d: 2D FFT operations for image and spatial data processing.
image: Image processing utilities (convolution, filtering, etc.)
image_ops: Image processing operations using 2D FFTs.
python: Python bindings for the spectrograms library.
source: Pluggable per-frame spectrogram sources.

Macros§

nzu

Structs§

Axes: Spectrogram axes container.
ChromaParams: Chroma feature parameters.
Chromagram: Chromagram representation with 12 pitch classes.
Complex: A complex number in Cartesian form.
CqtParams: CQT parameters
CqtResult: CQT result containing complex frequency bins and metadata.
ErbParams: ERB filterbank parameters
FftPlanner: A reusable FFT planner for efficient repeated FFT operations.
FrequencyAxis
InnerPlanner: A planner is used to create FFTs. It caches results internally, so when making more than one FFT it is advisable to reuse the same planner.
LogHzParams: Logarithmic frequency scale parameters
LogParams
MdctParams: Parameters for MDCT computation.
MelParams: Mel filter bank parameters
Mfcc: MFCC features representation.
MfccParams: MFCC computation parameters.
OverlapSaveConvolver: Streaming FFT convolution engine (overlap-save).
RealFftC2cPlan: Complex-to-complex FFT plan (f64) backed by rustfft.
RealFftInversePlan: Complex-to-Real Inverse FFT Plan
RealFftInversePlan2d
RealFftPlan: Real-to-Complex FFT Plan
RealFftPlan2d: 2D Real-to-Complex FFT Plan
RealFftPlanner: RealFftPlanner
Spectrogram: Spectrogram structure holding the computed spectrogram data and metadata.
SpectrogramParams: Spectrogram computation parameters.
SpectrogramParamsBuilder: Builder for SpectrogramParams.
SpectrogramPlan: A spectrogram plan is the compiled, reusable execution object.
SpectrogramPlanner: A planner is an object that can build spectrogram plans.
StftParams: STFT parameters for spectrogram computation.
StftParamsBuilder: Builder for StftParams.
StftPlan: STFT plan containing reusable FFT plan and buffers.
StftResult: STFT (Short-Time Fourier Transform) result containing complex frequency bins.

Enums§

ChromaNorm: Normalization strategy for chroma features.
Cqt: Constant-Q Transform frequency scale
Decibels: Decibel amplitude scale
Erb: ERB/gammatone frequency scale
ErbSpacing: Center-frequency spacing strategy for the ERB filterbank.
LinearHz: Linear frequency scale
LogHz: Logarithmic frequency scale
Magnitude: Magnitude amplitude scale
Mel: Mel frequency scale
MelNorm: Mel filterbank normalization strategy.
Power: Power amplitude scale
SpectrogramError: Represents errors that can occur in the spectrogram library.
WindowType: Window functions for spectral analysis and filtering.

Constants§

N_CHROMA: Number of pitch classes in Western music.

Traits§

AmpScaleSpec: Marker trait so we can specialise behaviour by AmpScale.
C2cPlan: A planned complex-to-complex FFT for a fixed transform length.
C2rPlan: A planned complex-to-real inverse FFT for a fixed transform length.
C2rPlanner: Planner that can construct inverse FFT plans.
ComplexToReal: An inverse FFT that takes a complex spectrum of length N/2+1 and transforms it to a real-valued signal of length N.
FftNum: Generic floating point number, implemented for f32 and f64
R2cPlan: A planned real-to-complex FFT for a fixed transform length.
R2cPlanner: Planner that can construct FFT plans.
RealToComplex: A forward FFT that takes a real-valued input signal of length N and transforms it to a complex spectrum of length N/2+1.
Sample: Sealed trait describing a float scalar the FFT backend can operate on.

Functions§

blackman_window
chromagram: Compute chromagram directly from audio samples.
chromagram_from_spectrogram: Compute chromagram from a magnitude or power spectrogram.
cqt: Compute the Constant-Q Transform (CQT) of a signal.
fft: Compute the real-to-complex FFT of a real-valued signal.
fft_convolve: Linear convolution of two real signals via FFT.
fft_deconvolve: Regularised spectral-division deconvolution of two real signals.
gammatone_center_frequencies: Build a gammatone magnitude spectrogram using the time-domain IIR filter bank.
gammatone_iir_spectrogram
gaussian_window
hamming_window
hanning_window
imdct: Compute the IMDCT (inverse MDCT) from MDCT coefficients.
irfft: Compute the inverse real FFT (complex-to-real IFFT).
istft: Reconstruct a time-domain signal from its STFT using overlap-add.
kaiser_window
magnitude_spectrum: Compute the magnitude spectrum of a signal (|X|).
make_window: Generate window function samples.
mdct: Compute the MDCT of an audio signal.
mfcc: Compute MFCCs directly from audio samples.
mfcc_from_log_mel: Compute MFCCs from a log mel spectrogram.
minimum_phase: Convert an FIR impulse response to its minimum-phase equivalent.
minimum_phase_with: Like minimum_phase but with explicit output length and oversampling factor.
power_spectrum: Compute the power spectrum of a signal (|X|²).
r2c_output_size: Output size for a real-to-complex FFT of length n.
rectangular_window
rfft: Compute the real-valued fft of a signal.
stft: Compute the Short-Time Fourier Transform (STFT) of a signal.