Crate polyvoice 

polyvoice

Speaker diarization library for Rust — online (streaming) and offline (file-based), ecosystem-agnostic. The ONNX path is opt-in (features = ["onnx", …]); default features are empty so BYO-embedder consumers can use pipeline::LegacyPipeline / StreamingPipeline / EnergyVad without linking ort.

Designed to be embedded into any Rust application that needs to answer the question “who spoke when?”.

Quick start

ONNX production path: the crate-root Pipeline (re-exported from pipeline_v2) + ModelRegistry, gated on features onnx, download, segmentation, embedder, clusterer, resegmentation (CLI also enables vbx). This is what CLI / FFI / Python / MCP run by default since 0.11 (v2 + VBx). With the gate off there is deliberately no crate-root Pipeline — ort-free builds use pipeline::LegacyPipeline.

Library mode (no ONNX): default-features = false, implement Embedder, pair with EnergyVad and pipeline::LegacyPipeline / streaming::StreamingPipeline — see the crate README and docs/library-mode.md.

Module organization

Two intentional pipeline families share math and types:

• ONNX production (pipeline_v2, crate-root Pipeline): trait-wired Segmenter → Embedder → Clusterer → Resegmenter. CLI/FFI/Python/MCP default since 0.11 (VBx when PLDA is available). See docs/PIPELINE-ARCHITECTURE.md.
• BYO / ort-free (pipeline::LegacyPipeline + StreamingPipeline): inject Embedder + VoiceActivityDetector. CLI --legacy uses this offline path with Silero + AHC.
• Shared math: ahc, kmeans, spectral, features, der, utils.
• Online centroids: production streaming uses streaming::ArrivalOrderSpeakerCache; cluster::SpeakerCluster is deprecated (not on any production path).

Re-exports

pub use asr::Asr;

pub use asr::AsrError;

pub use features::FbankConfig;

pub use features::FbankExtractor;

pub use utils::merge_segments;

pub use segmentation::AggregationConfig;

pub use segmentation::Aggregator;

pub use segmentation::FrameLabel;

pub use segmentation::MIN_AUDIO_SAMPLES;

pub use segmentation::PowersetClass;

pub use segmentation::PowersetDecoder;

pub use segmentation::RawSegment;

pub use segmentation::SegmentationError;

pub use segmentation::Segmenter;

pub use segmentation::WindowOutput;

pub use segmentation::PowersetConfig;

pub use segmentation::PowersetSegmenter;

pub use embedder::DummyExtractor;

pub use embedder::Embedder;

pub use embedder::EmbedderError;

pub use embedder::apply_overlap_mask;

pub use embedder::CamPlusPlusExtractor;

pub use embedder::ResNet34Adapter;

pub use clusterer::KMeansClusterer;Deprecated

pub use clusterer::AhcClusterer;

pub use clusterer::Clusterer;

pub use clusterer::ClustererError;

pub use clusterer::KmeansClusterer;

pub use clusterer::MinClusterSizeClusterer;

pub use clusterer::NmeScClusterer;

pub use clusterer::vbx::VbxClusterer;

pub use clusterer::vbx::VbxClustererConfig;

pub use resegmentation::OverlapRegionInput;

pub use resegmentation::OverlapResegmenter;

pub use resegmentation::ResegmentError;

pub use resegmentation::ResegmentInputs;

pub use resegmentation::Resegmenter;

pub use resegmentation::SpeakerCentroid;

pub use resegmentation::compute_centroids;

pub use resegmentation::extract_overlap_time_ranges;

pub use labeling::UncoveredPolicy;

pub use labeling::assign_speakers_by_midpoint;

pub use labeling::label_words;

pub use labeling::speaker_at;

pub use labeling::speaker_at_stable;

pub use attribution::AttributionConfig;

pub use attribution::SpeakerEmbedding;

pub use attribution::WhoSaidWhat;

pub use attribution::WordAnchor;

pub use attribution::attribute_and_fill;

pub use attribution::attribute_and_fill_with_config;

pub use attribution::attribute_words;

pub use attribution::attribute_words_with_config;

pub use attribution::fill_turn_text;

pub use attribution::fill_turn_text_with_config;

pub use attribution::interpolate_word_timestamps;

pub use attribution::speaker_embeddings_from_segments;

pub use attribution::who_said_what;

pub use attribution::who_said_what_with_config;

pub use pipeline_v2::Pipeline;

pub use pipeline_v2::PipelineConfig;

pub use pipeline_v2::PipelineError;

pub use vad::EnergyVad;

pub use vad::VadConfig;

pub use vad::VadError;

pub use vad::VoiceActivityDetector;

pub use vad::segment_speech;

pub use silero_vad::SileroVad;

pub use der::DerDecomposition;

pub use der::DerResult;

pub use der::SpeakerRecall;

pub use der::WderResult;

pub use der::compute_der;

pub use der::compute_wder;

pub use models::ModelRegistry;

pub use models::ProfileModels;

pub use models::RegistryError;

pub use overlap::OverlapRegion;

pub use types::ClusterConfig;

pub use types::Confidence;

pub use types::ConfigError;

pub use types::DEFAULT_AHC_THRESHOLD;

pub use types::DiarizationConfig;

pub use types::DiarizationResult;

pub use types::Profile;

pub use types::SampleRate;

pub use types::Segment;

pub use types::SpeakerId;

pub use types::SpeakerIdRemap;

pub use types::SpeakerSummary;

pub use types::SpeakerTurn;

pub use types::TimeRange;

pub use types::Transcript;

pub use types::Word;

pub use types::WordAlignment;

pub use types::confidence_from_distance;

pub use types::confidence_from_similarity;

pub use types::exclusive_turns;

pub use types::mean_speaker_embeddings;

pub use types::remap_segments;

pub use types::remap_turns;

pub use types::segment_confidences_from_embeddings;

pub use window::WindowBuffer;

pub use window::WindowIter;

pub use fbank_onnx::FbankOnnxExtractor;

Modules

ahc

Agglomerative Hierarchical Clustering (AHC) for speaker diarization.

asr

ASR (speech-to-text) trait — the stable interface the opt-in polyvoice-asr companion crate implements and the word→speaker join targets.

attribution

Word→speaker attribution: join raw ASR words to diarization turns.

clusterDeprecated

Online incremental speaker centroids. Kept for the fuzz target and experiments; not on any production path (offline clustering is clusterer::Clusterer, streaming uses streaming::ArrivalOrderSpeakerCache). Online incremental speaker centroids (SpeakerCluster).

clusterer

v1.0 Clusterer trait + concrete clusterers (NME-SC, AHC).

der

Diarization Error Rate (DER) and word-level attribution metrics.

embedder

Bring-your-own speaker embedder trait (always available; pure Rust core). ONNX-backed adapters still require features = ["onnx", "embedder"]. v1.0 Embedder trait + concrete extractors (CAM++, ResNet34, ERes2NetV2) + overlap-mask helper.

fbank_onnx

Shared fbank + ONNX speaker embedding engine (WeSpeaker, CAM++, ERes2Net, …).

features

Log-mel filterbank (fbank) feature extraction for speaker embeddings.

format

Subtitle / plain-text projections of a diarization result (SRT, WebVTT, TXT).

kmeans

K-Means++ clustering with automatic k selection via silhouette score.

labeling

Midpoint word→speaker labeling for STT stacks (always-on, no models). Lightweight word → speaker labeling for STT product consumers.

models

Model registry — manifest-driven downloads with SHA-256 verification, adapter selection by config string, and self-describing model metadata.

onnx

ONNX-based speaker embedding extractor with a session pool.

overlap

Overlap detection: identify frames where multiple speakers may be active.

pipeline

BYO / ort-free legacy pipeline (v1). The crate-root Pipeline is the production v2 pipeline (below) when its feature gate is on; with default features there is no crate-root Pipeline at all. High-level offline diarization pipeline (legacy v1).

pipeline_v2

polyvoice::pipeline_v2 — trait-wired production ONNX diarization pipeline.

resegmentation

v1.0 OverlapResegmenter — overlap-aware post-clustering pass.

rttm

RTTM (Rich Transcription Time Marked) parser and writer.

segmentation

Speaker segmentation: powerset-classifier + sliding-window aggregator.

silero_vad

Silero VAD (v6-generation) ONNX integration.

sortformer

Optional NVIDIA Streaming Sortformer v2 E2E diarizer (≤4 speakers). Opt-in via --features sortformer. See docs/sortformer.md. Optional NVIDIA Streaming Sortformer v2 end-to-end diarizer.

spectral

Spectral clustering math for speaker diarization.

streaming

Real-time streaming diarization pipeline.

types

Core types for speaker diarization.

utils

Math utilities for diarization.

vad

Voice Activity Detection (VAD) trait and utilities.

wav

Audio file I/O for the loading layer.

window

Sliding-window utilities for batch and streaming pipelines.