polyvoice

WAV in, speaker turns out.

A speaker diarization crate. Powerset neural segmentation, WeSpeaker ResNet34 embeddings, VBx clustering with automatic speaker count. One Pipeline call from 16 kHz mono to timestamped turns. The default build pulls no ONNX Runtime: hand-written INT8 kernels, ~8.4 MB production model pair, MIT, ungated. ONNX Runtime is a feature (cli-ort), not a requirement. Python, C FFI and a CLI ship from the same crate.

Examples

Library (kernels, models auto-download):

use polyvoice::models::ModelRegistry;
use polyvoice::types::{Profile, SampleRate};
use polyvoice::{Pipeline, PipelineConfig};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // PipelineConfig::default() is VBx when the `vbx` feature is on.
    let pipeline = Pipeline::builder()
        .config(PipelineConfig {
            profile: Profile::Balanced,
            ..PipelineConfig::default()
        })
        .with_models_from(ModelRegistry::default()?)
        .build()?;
    // 16 kHz WAV via ryf. Other rates/formats: `--features audio-io`.
    let (samples, sr) = polyvoice::wav::load_audio(std::path::Path::new("meeting.wav"))?;
    let result = pipeline.run(&samples, SampleRate::new(sr).ok_or("bad sample rate")?)?;
    for turn in &result.turns {
        println!("{}: {:.1}s - {:.1}s", turn.speaker, turn.time.start, turn.time.end);
    }
    Ok(())
}

CLI:

polyvoice download-models --profile balanced   # ~8.4 MB, MIT, no token
polyvoice diarize meeting.wav --output meeting.rttm

SPEAKER meeting 1   0.000  12.784  <NA> <NA> SPEAKER_00 <NA> <NA>
SPEAKER meeting 1  13.005   2.530  <NA> <NA> SPEAKER_01 <NA> <NA>
SPEAKER meeting 1  15.688  10.323  <NA> <NA> SPEAKER_02 <NA> <NA>

A 1-hour meeting diarizes in about a minute on a laptop. Python: pip install polyvoice (python/README.md). C FFI: docs/FFI.md.

Surfaces

Compared to pyannote

Like-for-like, strict collar 0, VoxConverse-test (232 files). Full matrix (incl. diart, whisperx, speakrs): compare.

The trade is explicit: ~4 DER points for a CPU-only, MIT, ungated deploy with no Python. Not the accuracy leader — the deployability leader.

Speed

Kernels (product default) vs same-host ONNX Runtime, EP=cpu, INT8. Linux x86_64: Ryzen AI 9 HX 370, 2026-09-08. Darwin: Apple Silicon. DER₀ is strict collar 0. Protocol: benchmarks.

Peak RSS on the Vox-3 smoke: ~310 MiB kernels vs ~620 MiB ort at jobs=1; ~470 MiB vs ~740 MiB at --jobs 3 (one shared pipeline, DER bit-identical to jobs=1). On-disk INT8 pair: 8,414,314 bytes — a locked scoreboard floor, as are DER and RSS (tests/native_scoreboard.json).

How it works

audio (f32 PCM)
  → powerset neural segmentation (overlap-aware)
  → WeSpeaker ResNet34 embeddings
  → VBx clustering (AHC / K-means / NME-SC alternatives, automatic speaker count)
  → overlap resegmentation → speaker turns

Install

[dependencies]
polyvoice = { version = "0.20", features = ["pipeline-native", "vbx"] }

rustc 1.94. Default features are empty: the published crate is the ort-free BYO core; models and engines are opt-in features (library mode).

benchmarks | api | architecture | library mode | ffi | python | production readiness | CHANGELOG

Batch diarization only: no ASR, no speaker identification. Beta (0.x): the public API may break between minor versions — pin an exact version in production. MIT.

Name: this project is polyvoice — speaker diarization for Rust, unrelated to ByteDance's "PolyVoice" speech-translation research.