Crate polyvoice 

polyvoice

Speaker diarization library for Rust — online (streaming) and offline (file-based), ecosystem-agnostic. Default features are empty so BYO-embedder consumers can use pipeline::LegacyPipeline / StreamingPipeline / EnergyVad without linking an inference runtime. The product path is hand-written INT8 kernels (pipeline-native / cli); there is no ONNX Runtime in this crate. Tract is opt-in (cli-tract / backend-tract).

Designed to be embedded into any Rust application that needs to answer the question “who spoke when?”. Freeze window and bump rules: docs/semver.md.

Quick start

Product path (CLI / FFI / MCP): crate-root Pipeline via pipeline-native (cli) — hand-written INT8 kernels, no libonnxruntime. Tract is cli-tract. The v2 gate is download + stage features plus an engine (pipeline-native or backend-tract). With the gate off there is deliberately no crate-root Pipeline — inference-free builds use pipeline::LegacyPipeline.

Library mode (no ONNX): default-features = false, implement Embedder, pair with EnergyVad and pipeline::LegacyPipeline / streaming::StreamingPipeline — see the crate README and docs/library-mode.md. WAVE ingest (wav) is always-on via ryf; mp3/flac/ogg/m4a need --features audio-io.

Module organization

Two intentional pipeline families share math and types:

• Production (pipeline_v2, crate-root Pipeline): trait-wired Segmenter → Embedder → Clusterer → Resegmenter. CLI/FFI/MCP default to hand-written kernels (cli). Tract is cli-tract. See docs/PIPELINE-ARCHITECTURE.md.
• BYO (pipeline::LegacyPipeline + StreamingPipeline): inject Embedder + VoiceActivityDetector. Product CLI does not run --legacy (no Silero runtime in this crate).
• Shared math: ahc, kmeans, spectral, features, der, utils.
• Online centroids: production streaming uses streaming::ArrivalOrderSpeakerCache.

Re-exports

pub use asr::Asr;

pub use asr::AsrError;

pub use features::FbankConfig;

pub use features::FbankExtractor;

pub use utils::merge_segments;

pub use segmentation::AggregationConfig;

pub use segmentation::Aggregator;

pub use segmentation::FrameLabel;

pub use segmentation::MIN_AUDIO_SAMPLES;

pub use segmentation::PowersetClass;

pub use segmentation::PowersetDecoder;

pub use segmentation::RawSegment;

pub use segmentation::SegmentationError;

pub use segmentation::Segmenter;

pub use segmentation::WindowOutput;

pub use segmentation::PowersetNative;

pub use embedder::DummyExtractor;

pub use embedder::Embedder;

pub use embedder::EmbedderError;

pub use embedder::apply_overlap_mask;

pub use embedder::ResNet34Native;

pub use clusterer::AhcClusterer;

pub use clusterer::AsNormClusterer;

pub use clusterer::Clusterer;

pub use clusterer::ClustererError;

pub use clusterer::KmeansClusterer;

pub use clusterer::MinClusterSizeClusterer;

pub use clusterer::NmeScClusterer;

pub use clusterer::vbx::VbxClusterer;

pub use clusterer::vbx::VbxClustererConfig;

pub use resegmentation::OverlapRegionInput;

pub use resegmentation::OverlapResegmenter;

pub use resegmentation::ResegmentError;

pub use resegmentation::ResegmentInputs;

pub use resegmentation::Resegmenter;

pub use resegmentation::SpeakerCentroid;

pub use resegmentation::compute_centroids;

pub use resegmentation::extract_overlap_time_ranges;

pub use labeling::UncoveredPolicy;

pub use labeling::assign_speakers_by_midpoint;

pub use labeling::label_words;

pub use labeling::speaker_at;

pub use labeling::speaker_at_stable;

pub use attribution::AttributionConfig;

pub use attribution::SpeakerEmbedding;

pub use attribution::WhoSaidWhat;

pub use attribution::WordAnchor;

pub use attribution::attribute_and_fill;

pub use attribution::attribute_and_fill_with_config;

pub use attribution::attribute_words;

pub use attribution::attribute_words_with_config;

pub use attribution::fill_turn_text;

pub use attribution::fill_turn_text_with_config;

pub use attribution::interpolate_word_timestamps;

pub use attribution::speaker_embeddings_from_segments;

pub use attribution::who_said_what;

pub use attribution::who_said_what_with_config;

pub use pipeline_v2::ClustererKind;

pub use pipeline_v2::Pipeline;

pub use pipeline_v2::PipelineBuilder;

pub use pipeline_v2::PipelineConfig;

pub use pipeline_v2::PipelineError;

pub use vad::EnergyVad;

pub use vad::VadConfig;

pub use vad::VadError;

pub use vad::VoiceActivityDetector;

pub use vad::segment_speech;

pub use der::DerDecomposition;

pub use der::DerResult;

pub use der::SpeakerRecall;

pub use der::WderResult;

pub use der::compute_der;

pub use der::compute_wder;

pub use models::ModelRegistry;

pub use models::ProfileModels;

pub use models::RegistryError;

pub use overlap::OverlapRegion;

pub use types::ClusterConfig;

pub use types::Confidence;

pub use types::ConfigError;

pub use types::DEFAULT_AHC_THRESHOLD;

pub use types::DiarizationConfig;

pub use types::DiarizationResult;

pub use types::Profile;

pub use types::SampleRate;

pub use types::Segment;

pub use types::SpeakerId;

pub use types::SpeakerIdRemap;

pub use types::SpeakerSummary;

pub use types::SpeakerTurn;

pub use types::TimeRange;

pub use types::Transcript;

pub use types::Word;

pub use types::WordAlignment;

pub use types::confidence_from_distance;

pub use types::confidence_from_similarity;

pub use types::exclusive_turns;

pub use types::mean_speaker_embeddings;

pub use types::remap_segments;

pub use types::remap_turns;

pub use types::segment_confidences_from_embeddings;

pub use window::WindowBuffer;

pub use window::WindowIter;

Modules

ahc

Agglomerative Hierarchical Clustering (AHC) for speaker diarization.

asr

ASR (speech-to-text) trait — the stable interface the opt-in polyvoice-asr companion crate implements and the word→speaker join targets.

attribution

Word→speaker attribution: join raw ASR words to diarization turns.

clusterer

v1.0 Clusterer trait + concrete clusterers (NME-SC, AHC).

der

Diarization Error Rate (DER) and word-level attribution metrics.

embedder

Bring-your-own speaker embedder trait (always available; pure Rust core). Tract-backed adapters (CamPlusPlusExtractor, ResNet34Adapter, …) require features = ["backend-tract", "embedder"]. Native ResNet34 is embedder-native. v1.0 Embedder trait + concrete extractors (CAM++, ResNet34, ERes2NetV2) + overlap-mask helper.

features

Log-mel filterbank (fbank) feature extraction for speaker embeddings.

format

Subtitle / plain-text projections of a diarization result (SRT, WebVTT, TXT).

kmeans

K-Means++ clustering with automatic k selection via silhouette score.

labeling

Midpoint word→speaker labeling for STT stacks (always-on, no models). Lightweight word → speaker labeling for STT product consumers.

models

Model registry — manifest-driven downloads with SHA-256 verification, adapter selection by config string, and self-describing model metadata.

overlap

Overlap detection: identify frames where multiple speakers may be active.

pipeline

BYO / ort-free legacy pipeline (v1). The crate-root Pipeline is the production v2 pipeline (below) when its feature gate is on; with default features there is no crate-root Pipeline at all. High-level offline diarization pipeline (legacy v1).

pipeline_v2

polyvoice::pipeline_v2 — trait-wired production diarization pipeline (kernels, ONNX Runtime, or tract).

resegmentation

v1.0 OverlapResegmenter — overlap-aware post-clustering pass.

rttm

RTTM (Rich Transcription Time Marked) parser and writer.

segmentation

Speaker segmentation: powerset-classifier + sliding-window aggregator.

spectral

Spectral clustering math for speaker diarization.

streaming

Real-time streaming diarization pipeline.

types

Core types for speaker diarization.

utils

Math utilities for diarization.

vad

Voice Activity Detection (VAD) trait and utilities.

wav

Audio file I/O for the loading layer.

window

Sliding-window utilities for batch and streaming pipelines.