Expand description
§rightkit-asr
Rust bindings for NVIDIA’s Parakeet speech recognition model using ONNX Runtime.
Parakeet is a state-of-the-art automatic speech recognition (ASR) model developed by NVIDIA, based on the FastConformer-TDT architecture with 600 million parameters.
§Features
- Easy-to-use API for speech-to-text transcription
- Support for ONNX format models
- 16kHz mono audio input
- Punctuation and capitalization included in output
- Fast inference using ONNX Runtime
§Quick Start
use rightkit_asr::{Parakeet, Transcriber, TimestampMode};
// Load the model
let mut parakeet = Parakeet::from_pretrained(".", None)?;
// Transcribe audio samples (see examples/raw.rs for audio loading)
let result = parakeet.transcribe_samples(audio, sample_rate, channels, Some(TimestampMode::Words))?;
println!("Transcription: {}", result.text);§Model Requirements
Your model directory should contain:
model.onnx- The ONNX model filemodel.onnx_data- External model weightsconfig.json- Model configurationpreprocessor_config.json- Audio preprocessing configurationtokenizer.json- Tokenizer vocabularytokenizer_config.json- Tokenizer configuration
§Audio Requirements
- Format: WAV
- Sample Rate: 16kHz
- Channels: Mono (stereo will be converted automatically)
- Bit Depth: 16-bit PCM or 32-bit float
Re-exports§
pub use platform::EngineSupport;pub use whisper::audio_duration;pub use whisper::is_unspaced_language;pub use whisper::is_whisper_language;pub use whisper::join_segment_text;pub use whisper::load_wav_mono_16k;pub use whisper::recording_deadline;pub use whisper::samples_to_ms;pub use whisper::validate_language;pub use whisper::CancelToken;pub use whisper::DecodeMode;pub use whisper::DecodeQuality;pub use whisper::InterruptReason;pub use whisper::Language;pub use whisper::LanguageDetection;pub use whisper::ProgressCallback;pub use whisper::ProgressProbe;pub use whisper::Whisper;pub use whisper::WhisperCpp;pub use whisper::WhisperInterrupted;pub use whisper::WhisperModel;pub use whisper::WhisperOptions;pub use whisper::WhisperProgress;pub use whisper::WhisperSegment;pub use whisper::WhisperTranscript;pub use whisper::WhisperWord;pub use whisper::RECORDING_DEADLINE_AUDIO_FACTOR;pub use whisper::RECORDING_DEADLINE_BASE;pub use whisper::UNSPACED_LANGUAGES;pub use whisper::WHISPER_LANGUAGES;pub use whisper::WHISPER_SAMPLE_RATE;pub use routing::select_engine;pub use routing::Availability;pub use routing::EngineKind;pub use routing::EngineSelection;pub use routing::RoutingPolicy;pub use chunked::transcribe_spans;pub use chunked::ChunkResult;pub use longform::pad_terminal_silence;pub use longform::samples_to_secs;pub use longform::LongFormOptions;pub use longform::LongFormProgress;pub use longform::PadPolicy;pub use longform::SeamPolicy;pub use longform::TimedExports;
Modules§
- chunked
- VAD-chunked decoding for any
Transcriber: decode each speech span on its own and re-time the result onto the source timeline. Span production (thresholds, padding, chunk size) isrightkit-vad’s job; this only applies spans to a decoder. - conditioning
- ASR-facing audio conditioning, moved from HeardRight (
heardright_core::audio_conditioning) so every app feeds Parakeet the same front end. HeardRight consumes this copy once it adopts rightkit-asr; until then the two must stay byte-identical (the Phase 6 parity gate checks it). - kws
- Streaming keyword spotting on a Zipformer2 transducer.
- longform
- The one long-form Parakeet route: padded-window schedule, quiet cut, seam
stitching, timed-piece bookkeeping, word grouping and SRT/VTT/JSON export.
Every Parakeet backend (CoreML on macOS, ORT elsewhere) feeds its
single-window decoder into
transcribe_padded_window; nothing else stitches windows. - parakeet_
core - Runtime-agnostic Parakeet TDT decoding primitives: beam search, context bias, detokenization (incl. N0 filler policy), score arithmetic and per-platform decode policy.
- platform
- Which engines this build target can host.
- routing
- Engine selection by language. Policy-free mechanics: the caller states the language and which engines it has models for; this says which to use.
- whisper
- Whisper decoding behind the shared
Transcriberinterface.
Structs§
- Beam
Options - Beam decode options for the ONNX path. Defaults are the Windows / ONNX production policy (width 2, emission reward 0.10, duration-index cap 2, 10 symbols per frame, canonical N0 detokenizer).
- Execution
Config - Language
Transcription - A
TranscriptionResulttogether with the language the engine reports for it. Returned byTranscriber::transcribe_samples_with_languageso a caller reads the language from the result on every backend (macOS CoreML included) instead of assuming one.resultis exactly whattranscribe_samplesreturns. - Model
Config Json - Nemotron
- Nemotron streaming ASR model (0.6B parameters). We dont apply mel normalization unlike others…
- Nemotron
Encoder Cache - Encoder cache state for Nemotron streaming inference.
- Nemotron
Handle - Shared handle to a loaded Nemotron model. ONNX session is only loaded once and reference counted.
- Nemotron
Model - Nemotron ONNX wrapper. we handle encoder and decoder_joint sessions separately.
- Nemotron
Model Config - cfg for Nemotron model dims.
- Parakeet
- Parakeet
Decoder - ParakeetEOU
- Parakeet RealTime EOU model for streaming ASR with end-of-utterance detection. Uses cache-aware streaming with audio buffering for pre-encode context.
- ParakeetEOU
Handle - Shared handle to a loaded ParakeetEOU model. The ONNX session is loaded once and reference-counted.
- ParakeetEOU
Model - Parakeet
Model - ParakeetTDT
- Parakeet TDT model for multilingual ASR
- Parakeet
Unified - Parakeet
Unified Handle - Shared handle to a loaded ParakeetUnified model. The ONNX session is loaded once and reference-counted.
- Parakeet
Unified Model - Preprocessor
Config - Reported
Language - Language an engine reports for its output, a sibling of
TranscriptionResultso that struct keeps its fields.codeisNoneexactly whensourceisLanguageSource::Unknown. - Sentence
Piece Vocab - Minimal SentencePiece vocabulary loader. Parses the protobuf .model file to extract token strings. Note that, our vocab.rs cannot parse protobuf format. I haven’t test it with digit spacing yet, at least for this initial impl.
- Strict
Layout - What a strict load demands of a model directory. Mirrors the strict CoreML layout: each flag removes one silent fallback or surprise.
- Timed
Piece - One emitted model symbol with exact sample positions. Sibling of
TimedToken, which is unchanged and still returned by every existing method. Built from the decoder’s frame indices BEFORE detokenisation, so it is not merged or re-spaced: CutRight-style word-edge cuts can usestart_sample/end_sampledirectly. - Timed
Token - Timed
Word - A word (one or more pieces joined) with its timing, in milliseconds. The
suite-wide type, owned by
rightkit-transcript. One spoken word (end_msexclusive). - Transcription
Result - Transducer
- Transducer
Options - Unified
Model Config - Unified
Streaming Config - WavSpec
- Specifies properties of the audio data.
Enums§
- Audio
Source - Audio handed to a transcriber. WAV stays supported in-crate; every other
container or codec is the caller’s job: decode it to mono 16 kHz
f32in[-1, 1]and passAudioSource::Pcm16k. This crate does not depend on ffmpeg or any other decoder. - Error
- Execution
Provider - Language
Source - Where a
ReportedLanguagecame from. - Timestamp
Mode - Timestamp output mode for transcription results
- WavLoad
Error - Why
load_wav_mono_16k_headrefused a file. Each variant says what the file holds, so a host can tell the user what to convert.
Constants§
- LAST_
TOKEN_ MIN_ DURATION_ MS - Width, in milliseconds, given to a last token whose predicted duration is zero.
- UNSUPPORTED_
PREFIX - Prefix of the message of every
Error::unsupportederror.
Traits§
- Transcriber
- Trait for common transcription functionality
Functions§
- load_
audio - Read a WAV file into interleaved
f32samples in[-1, 1]plus itsWavSpec(sample rate, channel count, format). - load_
wav_ mono_ 16k_ head - Read at most the first
max_durationof a WAV file as mono 16 kHzf32in[-1, 1], for example to identify the language before decoding the whole recording. - process_
timestamps - Convert token timestamps to the requested output mode