speakrs

Fast Rust speaker diarization with pyannote-level accuracy.

On VoxConverse dev, speakrs CoreML gets 7.1% DER at 529x realtime versus pyannote's 7.2% at 24x. Full results are in benchmarks/.

If you want a small end-to-end app using it, see avencera/smrze.

Fast Rust speaker diarization.

speakrs implements the full pyannote community-1 style pipeline in Rust: segmentation, powerset decode, overlap-add aggregation, binarization, embedding, PLDA, and VBx clustering. There is no Python runtime in the library path. Inference runs on ONNX Runtime or native CoreML and the rest of the pipeline stays in Rust.

The goal is to get pyannote-class diarization without shipping a Python stack. On VoxConverse dev, speakrs CoreML gets 7.1% DER at 529x realtime versus pyannote's 7.2% at 24x. Full tables are in benchmarks/.

Usage

# macOS (CoreML)
speakrs = { version = "0.4", features = ["coreml"] }

# NVIDIA GPU
speakrs = { version = "0.4", features = ["cuda"] }

# CPU only
speakrs = "0.4"

# System OpenBLAS
speakrs = { version = "0.4", default-features = false, features = ["online", "openblas-system"] }

Quick start

use speakrs::{ExecutionMode, OwnedDiarizationPipeline};

fn main() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
    let mut pipeline = OwnedDiarizationPipeline::from_pretrained(ExecutionMode::CoreMl)?;

    let audio: Vec<f32> = load_your_mono_16khz_audio_here();
    let result = pipeline.run(&audio)?;

    print!("{}", result.rttm("my-audio"));
    Ok(())
}

Speaker turns

use speakrs::pipeline::{FRAME_DURATION_SECONDS, FRAME_STEP_SECONDS};

let result = pipeline.run(&audio)?;

for segment in result
    .discrete_diarization
    .to_segments(FRAME_STEP_SECONDS, FRAME_DURATION_SECONDS)
{
    println!("{:.3} - {:.3}  {}", segment.start, segment.end, segment.speaker);
}

Background queue

[QueueSender] and [QueueReceiver] run a background worker. Push audio from any thread and read results as they finish:

use speakrs::{ExecutionMode, OwnedDiarizationPipeline, QueuedDiarizationRequest};

let pipeline = OwnedDiarizationPipeline::from_pretrained(ExecutionMode::CoreMl)?;
let (tx, rx) = pipeline.into_queued()?;

std::thread::spawn(move || {
    for (file_id, audio) in receive_files() {
        tx.push(QueuedDiarizationRequest::new(file_id, audio)).unwrap();
    }
});

for result in rx {
    let result = result?;
    print!("{}", result.result?.rttm(&result.file_id));
}

Local models

For offline or airgapped setups, load models from a local directory:

use std::path::Path;
use speakrs::{ExecutionMode, OwnedDiarizationPipeline};

let mut pipeline = OwnedDiarizationPipeline::from_dir(
    Path::new("/path/to/models"),
    ExecutionMode::Cpu,
)?;
let result = pipeline.run(&audio)?;

Choosing a mode

The *-fast modes use a 2 second step instead of 1 second. They usually trade some boundary precision for more throughput. Start with coreml or cuda unless you already know you want the faster step size.

Benchmarks

VoxConverse dev, collar=0ms:

On VoxConverse test, both coreml and cuda match pyannote at 11.1% DER and are much faster. See benchmarks/ for the full tables across all datasets.

CoreML and ONNX Runtime can differ slightly even in FP32 because the runtime graphs are not identical and floating-point reduction order changes rounding.

Why not pyannote-rs?

pyannote-rs is the main Rust-only comparison point, but it targets a different tradeoff.

On the VoxConverse dev subset where pyannote-rs emits output, speakrs CoreML scores 11.5% DER versus 80.2% for pyannote-rs. In that same run, pyannote-rs returned no segments on most files.

Models

With the default online feature, models download on first use from avencera/speakrs-models. Set SPEAKRS_MODELS_DIR if you want to force a local bundle instead.

Features and build notes

Common features:

• online (default): model download via [ModelManager]
• coreml: native CoreML backend on macOS
• cuda: NVIDIA CUDA backend via ONNX Runtime
• load-dynamic: load the CUDA runtime at startup instead of static linking

BLAS backends matter if you disable default features:

• x86_64 defaults to statically linked Intel MKL
• non-x86_64 defaults to statically linked OpenBLAS and needs a C toolchain
• no-default builds must enable exactly one of intel-mkl, openblas-static, or openblas-system

speakrs = { version = "0.4", default-features = false, features = ["online", "intel-mkl"] }
speakrs = { version = "0.4", default-features = false, features = ["online", "openblas-system"] }

The ONNX Runtime dependency (ort 2.0.0-rc.12) is still pre-release.

Public API

Start here:

• [OwnedDiarizationPipeline]: pipeline entry point
• [QueueSender] and [QueueReceiver]: background worker interface
• [DiarizationResult]: frame-level activations, segments, clusters, embeddings, RTTM
• [PipelineConfig] and [RuntimeConfig]: tuning knobs
• [ModelManager]: model download when online is enabled
• [Segment]: a single speaker turn

Contributing

See CONTRIBUTING.md for local setup, model downloads, fixture generation, and the standard check commands used in this repo.

References

• pyannote-audio - Python reference implementation
• pyannote community-1 - VBx + PLDA pipeline
• SpeakerKit - Swift reference (same VBx architecture)