Crate audio_blocks

audio-blocks

This crate provides traits for audio blocks to generalize common problems in handling audio data, like different channel layouts, adapting between them and receiving varying number of samples. You will get Interleaved, Sequential and Stacked blocks and you can select where the data is stored by choosing between owned data, views and mutable views. Owned blocks will store the data on the heap, while views can be created over slices, raw pointers or from any other block. All of them implement the AudioBlock / AudioBlockMut traits and the mutable blocks implement operations.

This crate can be used in no_std contexts when disabling the default features. Owned blocks are stored on the heap and thus need either the alloc or the std feature. Everything in this library, except for generating new owned blocks, is real-time safe.

The main problem this crate is solving is that audio data can have different formats:

• Interleaved: [ch0, ch1, ch0, ch1, ch0, ch1]

  • Interpretation: Each group of channel samples represents a frame. So, this layout stores frames one after another.
  • Terminology: Described as “packed” or “frames first” because each time step is grouped and processed as a unit (a frame).
  • Usage: Often used in APIs or hardware-level interfaces, where synchronized playback across channels is crucial.

• Sequential: [ch0, ch0, ch0, ch1, ch1, ch1]

  • Interpretation: All samples from ch0 are stored first, followed by all from ch1, etc.
  • Terminology: Described as “planar” or “channels first” in the sense that all data for one channel appears before any data for the next.
  • Usage: Used in DSP pipelines where per-channel processing is easier and more efficient.

• Stacked: [[ch0, ch0, ch0], [ch1, ch1, ch1]]

  • Interpretation: Each channel has its own separate buffer or array.
  • Terminology: Also described as “planar” or “channels first” though more specifically it’s channel-isolated buffers.
  • Usage: Very common in real-time DSP, as it simplifies memory access and can improve SIMD/vectorization efficiency.

So if you write your processor functions expecting an impl [AudioBlock]<S> you can receive any kind of audio data, no matter which layout the audio API is using. AudioBlocks can contain any type of sample that is Copy, Default and 'static which is true for all kinds of numbers.

As you mostly don’t want your process function to work with any kind of number type, you can write a specialized process block, expecting only f32 samples.

fn process(block: &mut impl AudioBlockMut<f32>) {
    for channel in block.channels_mut() {
        for sample in channel {
            *sample *= 0.5;
        }
    }
}

or you can write a generic process block which works on all floating point values (f32, f64 and optionally half::f16) by using the Float trait from the num or num_traits crate:

use num_traits::Float;

fn process<F: Float>(block: &mut impl AudioBlockMut<F>) {
    let gain = F::from(0.5).unwrap();
    for channel in block.channels_mut() {
        for sample in channel {
            *sample *= gain;
        }
    }
}

Access to the audio data can be achieved with the iterators AudioBlock::channels() or AudioBlock::frames(), by accessing a specific one with AudioBlock::channel() or AudioBlock::frame() and accessing only one value with AudioBlock::sample(). Iterating over frames can be faster for interleaved data, while iterating over channels is always faster for sequential or stacked data.

All Trait Functions

AudioBlock

fn num_channels(&self) -> u16;
fn num_frames(&self) -> usize;
fn num_channels_allocated(&self) -> u16;
fn num_frames_allocated(&self) -> usize;
fn sample(&self, channel: u16, frame: usize) -> S;
fn channel(&self, channel: u16) -> impl Iterator<Item = &S>;
fn channels(&self) -> impl Iterator<Item = impl Iterator<Item = &S> + '_> + '_;
fn frame(&self, frame: usize) -> impl Iterator<Item = &S>;
fn frames(&self) -> impl Iterator<Item = impl Iterator<Item = &S> + '_> + '_;
fn view(&self) -> impl AudioBlock<S>;
fn layout(&self) -> BlockLayout;
fn raw_data(&self, stacked_ch: Option<u16>) -> &[S];

AudioBlockMut

contains all of the non-mutable functions plus:

fn resize(&mut self, num_channels: u16, num_frames: usize);
fn sample_mut(&mut self, channel: u16, frame: usize) -> &mut S;
fn channel_mut(&mut self, channel: u16) -> impl Iterator<Item = &mut S>;
fn channels_mut(&mut self) -> impl Iterator<Item = impl Iterator<Item = &mut S> + '_> + '_;
fn frame_mut(&mut self, frame: usize) -> impl Iterator<Item = &mut S>;
fn frames_mut(&mut self) -> impl Iterator<Item = impl Iterator<Item = &mut S> + '_> + '_;
fn view_mut(&mut self) -> impl AudioBlockMut<S>;
fn raw_data_mut(&mut self, stacked_ch: Option<u16>) -> &mut [S];

Operations

There are multiple operations defined on audio blocks, which allow copying data between them and applying an operation on each sample.

fn copy_from_block(&mut self, block: &impl AudioBlock<S>);
fn copy_from_block_resize(&mut self, block: &impl AudioBlock<S>);
fn for_each(&mut self, f: impl FnMut(&mut S));
fn for_each_including_non_visible(&mut self, f: impl FnMut(&mut S));
fn enumerate(&mut self, f: impl FnMut(u16, usize, &mut S));
fn enumerate_including_non_visible(&mut self, f: impl FnMut(u16, usize, &mut S));
fn fill_with(&mut self, sample: S);
fn clear(&mut self);

Create Audio Blocks

Owned

Types:

• Interleaved
• Sequential
• Stacked

fn new(num_channels: u16, num_frames: usize) -> Self;
fn from_block(block: &impl AudioBlock<S>) -> Self;

Views

Types:

• InterleavedView / InterleavedViewMut
• SequentialView / SequentialViewMut
• StackedView / StackedViewMut

fn from_slice(data: &'a [S], num_channels: u16, num_frames: usize) -> Self;
fn from_slice_limited(data: &'a [S], num_channels_visible: u16, num_frames_visible: usize, num_channels_allocated: u16, num_frames_allocated: usize) -> Self;

Interleaved and sequential blocks can be directly generated from pointers:

unsafe fn from_ptr(data: *const S, num_channels: u16, num_frames: usize) -> Self;
unsafe fn from_ptr_limited(data: *const S, num_channels_visible: u16, num_frames_visible: usize, num_channels_allocated: u16, num_frames_allocated: usize) -> Self;

Stacked blocks can only be generated from pointers using StackedPtrAdapter:

let mut adapter = unsafe { StackedPtrAdapter::<_, 16>::from_ptr(data, num_channels, num_frames) };
let block = adapter.stacked_view();

Handling Varying Number of Frames

The number of samples in audio buffers coming from audio APIs can vary with each process call and often only the maximum number of frames is given. This is the reason why all blocks have a number of allocated frames and channels and visible frames and channels.

With the function AudioBlockMut::resize() the buffers can be resized as long as they do not grow larger than the allocated memory. Resize is always real-time safe! When using the Ops::copy_from_block_resize() function, the destination block will automatically adapt the size of the source block. For views the from_slice_limited or from_ptr_limited functions will provide you with a way to directly limit the visible data of the underlying memory.

Here you see how you adapt your block size to incoming blocks with changing sizes if you need to copy the data for any reason:

fn process(&mut self, other_block: &mut impl AudioBlock<f32>) {
    self.block.copy_from_block_resize(other_block);
}

Performance Optimizations

The most performant way to iterate over blocks will be by accessing the raw data. But this can be dangerous because in limited blocks it will retrieve samples that are not meant to be visible and you have to figure out the data layout. For simple operations that do not depend on other samples like applying a gain, doing so for all samples, can be faster (if the amount of invisible samples is not exceptionally high). In the AudioBlockMut trait you will find Ops::for_each() and Ops::for_each_including_non_visible() or Ops::enumerate() and Ops::enumerate_including_non_visible() for this reason.

If you use the iterators AudioBlock::channels() or AudioBlock::frames() it depends on the layout which one will be more performant. For Sequential and Stacked it will be always faster to iterate over the channels, for Interleaved with higher channel counts it can be faster to iterate over frames.

The layout of a block can be retrieved using the AudioBlock::layout() function.

Re-exports

pub use ops::Ops;

pub use interleaved::Interleaved;

pub use interleaved::InterleavedView;

pub use interleaved::InterleavedViewMut;

pub use sequential::Sequential;

pub use sequential::SequentialView;

pub use sequential::SequentialViewMut;

pub use stacked::Stacked;

pub use stacked::StackedPtrAdapter;

pub use stacked::StackedPtrAdapterMut;

pub use stacked::StackedView;

pub use stacked::StackedViewMut;

Modules

interleaved

ops

sequential

stacked

Enums

BlockLayout

Represents the memory layout of audio data returned by AudioBlock::layout.

Traits

AudioBlock

Core trait for audio data access operations across various memory layouts.

AudioBlockMut

Extends the AudioBlock trait with mutable access operations.

Sample

Represents a sample type that can be stored and processed in audio blocks.