Skip to main content

Crate rightkit_asr

Crate rightkit_asr 

Source
Expand description

§rightkit-asr

Rust bindings for NVIDIA’s Parakeet speech recognition model using ONNX Runtime.

Parakeet is a state-of-the-art automatic speech recognition (ASR) model developed by NVIDIA, based on the FastConformer-TDT architecture with 600 million parameters.

§Features

  • Easy-to-use API for speech-to-text transcription
  • Support for ONNX format models
  • 16kHz mono audio input
  • Punctuation and capitalization included in output
  • Fast inference using ONNX Runtime

§Quick Start

use rightkit_asr::{Parakeet, Transcriber, TimestampMode};

// Load the model
let mut parakeet = Parakeet::from_pretrained(".", None)?;

// Transcribe audio samples (see examples/raw.rs for audio loading)
let result = parakeet.transcribe_samples(audio, sample_rate, channels, Some(TimestampMode::Words))?;
println!("Transcription: {}", result.text);

§Model Requirements

Your model directory should contain:

  • model.onnx - The ONNX model file
  • model.onnx_data - External model weights
  • config.json - Model configuration
  • preprocessor_config.json - Audio preprocessing configuration
  • tokenizer.json - Tokenizer vocabulary
  • tokenizer_config.json - Tokenizer configuration

§Audio Requirements

  • Format: WAV
  • Sample Rate: 16kHz
  • Channels: Mono (stereo will be converted automatically)
  • Bit Depth: 16-bit PCM or 32-bit float

Re-exports§

pub use platform::EngineSupport;
pub use whisper::audio_duration;
pub use whisper::is_unspaced_language;
pub use whisper::is_whisper_language;
pub use whisper::join_segment_text;
pub use whisper::load_wav_mono_16k;
pub use whisper::recording_deadline;
pub use whisper::samples_to_ms;
pub use whisper::validate_language;
pub use whisper::CancelToken;
pub use whisper::DecodeMode;
pub use whisper::DecodeQuality;
pub use whisper::InterruptReason;
pub use whisper::Language;
pub use whisper::LanguageDetection;
pub use whisper::ProgressCallback;
pub use whisper::ProgressProbe;
pub use whisper::Whisper;
pub use whisper::WhisperCpp;
pub use whisper::WhisperInterrupted;
pub use whisper::WhisperModel;
pub use whisper::WhisperOptions;
pub use whisper::WhisperProgress;
pub use whisper::WhisperSegment;
pub use whisper::WhisperTranscript;
pub use whisper::WhisperWord;
pub use whisper::RECORDING_DEADLINE_AUDIO_FACTOR;
pub use whisper::RECORDING_DEADLINE_BASE;
pub use whisper::UNSPACED_LANGUAGES;
pub use whisper::WHISPER_LANGUAGES;
pub use whisper::WHISPER_SAMPLE_RATE;
pub use routing::select_engine;
pub use routing::Availability;
pub use routing::EngineKind;
pub use routing::EngineSelection;
pub use routing::RoutingPolicy;
pub use chunked::transcribe_spans;
pub use chunked::ChunkResult;
pub use longform::pad_terminal_silence;
pub use longform::samples_to_secs;
pub use longform::LongFormOptions;
pub use longform::LongFormProgress;
pub use longform::PadPolicy;
pub use longform::SeamPolicy;
pub use longform::TimedExports;

Modules§

chunked
VAD-chunked decoding for any Transcriber: decode each speech span on its own and re-time the result onto the source timeline. Span production (thresholds, padding, chunk size) is rightkit-vad’s job; this only applies spans to a decoder.
conditioning
ASR-facing audio conditioning, moved from HeardRight (heardright_core::audio_conditioning) so every app feeds Parakeet the same front end. HeardRight consumes this copy once it adopts rightkit-asr; until then the two must stay byte-identical (the Phase 6 parity gate checks it).
kws
Streaming keyword spotting on a Zipformer2 transducer.
longform
The one long-form Parakeet route: padded-window schedule, quiet cut, seam stitching, timed-piece bookkeeping, word grouping and SRT/VTT/JSON export. Every Parakeet backend (CoreML on macOS, ORT elsewhere) feeds its single-window decoder into transcribe_padded_window; nothing else stitches windows.
parakeet_core
Runtime-agnostic Parakeet TDT decoding primitives: beam search, context bias, detokenization (incl. N0 filler policy), score arithmetic and per-platform decode policy.
platform
Which engines this build target can host.
routing
Engine selection by language. Policy-free mechanics: the caller states the language and which engines it has models for; this says which to use.
whisper
Whisper decoding behind the shared Transcriber interface.

Structs§

BeamOptions
Beam decode options for the ONNX path. Defaults are the Windows / ONNX production policy (width 2, emission reward 0.10, duration-index cap 2, 10 symbols per frame, canonical N0 detokenizer).
ExecutionConfig
LanguageTranscription
A TranscriptionResult together with the language the engine reports for it. Returned by Transcriber::transcribe_samples_with_language so a caller reads the language from the result on every backend (macOS CoreML included) instead of assuming one. result is exactly what transcribe_samples returns.
ModelConfigJson
Nemotron
Nemotron streaming ASR model (0.6B parameters). We dont apply mel normalization unlike others…
NemotronEncoderCache
Encoder cache state for Nemotron streaming inference.
NemotronHandle
Shared handle to a loaded Nemotron model. ONNX session is only loaded once and reference counted.
NemotronModel
Nemotron ONNX wrapper. we handle encoder and decoder_joint sessions separately.
NemotronModelConfig
cfg for Nemotron model dims.
Parakeet
ParakeetDecoder
ParakeetEOU
Parakeet RealTime EOU model for streaming ASR with end-of-utterance detection. Uses cache-aware streaming with audio buffering for pre-encode context.
ParakeetEOUHandle
Shared handle to a loaded ParakeetEOU model. The ONNX session is loaded once and reference-counted.
ParakeetEOUModel
ParakeetModel
ParakeetTDT
Parakeet TDT model for multilingual ASR
ParakeetUnified
ParakeetUnifiedHandle
Shared handle to a loaded ParakeetUnified model. The ONNX session is loaded once and reference-counted.
ParakeetUnifiedModel
PreprocessorConfig
ReportedLanguage
Language an engine reports for its output, a sibling of TranscriptionResult so that struct keeps its fields. code is None exactly when source is LanguageSource::Unknown.
SentencePieceVocab
Minimal SentencePiece vocabulary loader. Parses the protobuf .model file to extract token strings. Note that, our vocab.rs cannot parse protobuf format. I haven’t test it with digit spacing yet, at least for this initial impl.
StrictLayout
What a strict load demands of a model directory. Mirrors the strict CoreML layout: each flag removes one silent fallback or surprise.
TimedPiece
One emitted model symbol with exact sample positions. Sibling of TimedToken, which is unchanged and still returned by every existing method. Built from the decoder’s frame indices BEFORE detokenisation, so it is not merged or re-spaced: CutRight-style word-edge cuts can use start_sample / end_sample directly.
TimedToken
TimedWord
A word (one or more pieces joined) with its timing, in milliseconds. The suite-wide type, owned by rightkit-transcript. One spoken word (end_ms exclusive).
TranscriptionResult
Transducer
TransducerOptions
UnifiedModelConfig
UnifiedStreamingConfig
WavSpec
Specifies properties of the audio data.

Enums§

AudioSource
Audio handed to a transcriber. WAV stays supported in-crate; every other container or codec is the caller’s job: decode it to mono 16 kHz f32 in [-1, 1] and pass AudioSource::Pcm16k. This crate does not depend on ffmpeg or any other decoder.
Error
ExecutionProvider
LanguageSource
Where a ReportedLanguage came from.
TimestampMode
Timestamp output mode for transcription results
WavLoadError
Why load_wav_mono_16k_head refused a file. Each variant says what the file holds, so a host can tell the user what to convert.

Constants§

LAST_TOKEN_MIN_DURATION_MS
Width, in milliseconds, given to a last token whose predicted duration is zero.
UNSUPPORTED_PREFIX
Prefix of the message of every Error::unsupported error.

Traits§

Transcriber
Trait for common transcription functionality

Functions§

load_audio
Read a WAV file into interleaved f32 samples in [-1, 1] plus its WavSpec (sample rate, channel count, format).
load_wav_mono_16k_head
Read at most the first max_duration of a WAV file as mono 16 kHz f32 in [-1, 1], for example to identify the language before decoding the whole recording.
process_timestamps
Convert token timestamps to the requested output mode

Type Aliases§

Result