mozu-audio 0.3.0

• Japanese speech recognition with Haas.
• Long-audio transcription with incremental WAV reading and bounded audio buffering.
• Streaming sessions with provisional, confirmed, and final transcript updates.
• Silero VAD, microphone capture, resampling, token timestamps, and cancellation.
• Models stay loaded across recognition calls. Audio is processed locally.

At a Glance

Measured with Mozu Audio 0.1.0 on Apple M3 (macOS 26.6.2), using INT8 encoder weights and the FP32 split decoder. Read speech uses 479 clips from the Japanese FLEURS test split (91 minutes), with speed based on the mean of two passes. Long-form transcription uses quiet-point splitting on the nine J-CRe3 test recordings (23 minutes), with speed based on the sum of per-recording medians over four passes. RTFx is audio duration divided by processing time. Timings exclude model loading, warm-up, and file I/O. CER is character error rate after text normalization.

Requirements

• macOS on Apple Silicon.
• Rust 1.94 or later.
• Compatible Haas Core ML models and preprocessing/tokenizer assets.
• A Silero VAD Core ML model for speech-boundary detection.

Add the library

Add the SDK from crates.io:

cargo add mozu-audio@0.3.0

To use a local checkout:

cargo add mozu-audio --path /path/to/mozu-audio/crates/core

Models

Use ModelCache to download the compiled Haas Core ML models and Silero VAD into an application-owned directory. A verified cache works offline. Call these blocking methods on a worker thread in GUI applications.

use mozu_audio::{Cancellation, ComputeConfig, HaasEngine, models::ModelCache};
use std::path::PathBuf;

fn load_haas(cache_dir: PathBuf) -> mozu_audio::Result<HaasEngine> {
    let cache = ModelCache::new(cache_dir);
    let files = cache.prepare_haas(&Cancellation::new(), |progress| {
        eprintln!("{}: {} bytes", progress.file, progress.received_bytes);
    })?;
    files.load(ComputeConfig::standard())
}

For VAD, pass the path from cache.prepare_silero(&cancellation, |_| {})? to SileroVad::load. To use the local-path WAV example below, place assets/, encoder.mlmodelc/, and predictor.mlmodelc/ under models/haas/.

Use HaasEngine::load for a fused predictor/joint. For separate models, use HaasEngine::load_split with a prediction model and either joint.mlmodelc (full logits) or joint-decision.mlmodelc (scalar decisions).

Model support currently covers Haas-ja-v1.0 and Silero VAD 6.2.1. Original PyTorch checkpoints cannot be passed directly to the runtime. The exporters are tools/convert_haas_coreml.py and tools/convert_silero_coreml.py; run them with --help for conversion requirements and arguments. Python is needed only when exporting models.

Transcribe a WAV file

use mozu_audio::{Cancellation, HaasEngine};
use std::path::Path;

fn main() -> mozu_audio::Result<()> {
    let engine = HaasEngine::load(
        Path::new("models/haas/assets"),
        Path::new("models/haas/encoder.mlmodelc"),
        Path::new("models/haas/predictor.mlmodelc"),
    )?;
    let result = engine.transcribe_wav(
        Path::new("speech.wav"),
        &Cancellation::new(),
    )?;
    println!("{}", result.text);
    Ok(())
}

For in-memory audio, use HaasEngine::transcribe_samples with 16 kHz mono f32 samples. A single call accepts at least 0.64 seconds and up to the loaded model's limit. Read the sample bounds with engine.input_limits(). Use the batch or streaming API for longer input. Keep the engine on the worker thread that created it and reuse it across calls.

Long-audio transcription

Use batch::transcribe_quiet_file to transcribe a recording in ranges split at quiet points:

use mozu_audio::{Cancellation, HaasEngine, batch};
use std::path::Path;

fn transcribe_file(engine: &HaasEngine, path: &Path) -> mozu_audio::Result<String> {
    let result = batch::transcribe_quiet_file(
        engine,
        path,
        &Cancellation::new(),
        |progress| {
            println!("{} / {} samples", progress.processed_samples, progress.total_samples);
            Ok(())
        },
    )?;
    Ok(result.transcript.text)
}

Each range fits the loaded model. Batch results include the transcript, token timestamps, elapsed processing time, and peak audio buffer size. Use transcribe_quiet_wav for canonical 16 kHz mono float32 or PCM16 WAV input, or transcribe_quiet_source for a reusable audio source. The _result variants return shared ASR metadata and token pieces. For VAD-based overlapping windows, use batch::transcribe_file with a loaded SileroVad and AsrConfig.

Use transcribe_quiet_file_with_updates or transcribe_file_with_updates to receive partial text, tokens, and processing progress. The final update contains the assembled transcript.

Streaming recognition

Create a streaming::StreamingSession with a loaded engine, VAD model, and AsrConfig::interactive_for(engine.input_limits())?. Feed 16 kHz mono f32 audio through push_audio, with at most 16,000 samples per call, and consume the returned updates before sending the next block.

Apply token revisions by retaining retained_token_count tokens and appending appended_tokens. Tokens before stable_token_count are immutable. Process empty revisions as well; a final update makes every remaining token stable.

Call finish to flush the remaining input and retrieve the final transcript, or cancel to discard the session. Call reset before reusing a completed, cancelled, or failed session. Loaded models are retained. Microphone capture and native-rate-to-16 kHz resampling are available in recording.

Use AsrService::load_windowed_with_options or AsrService::load_with_options with a source_capacity greater than one to share loaded models across independent sources. Start a session for each SourceId and use snapshot to read its current transcript.

Select allowed Core ML compute units for the encoder and decoder with ComputeConfig. Pass it to the engine's load_with_compute or load_split_with_compute, or set it in ServiceOptions. Use SileroVad::load_with_compute to select VAD compute units.

Streaming uses overlapping windows over the offline Haas model. It does not provide model-native cached streaming.

Custom vocabulary

Create VocabularyTerm entries with the desired spelling and optional readings. Load the split decoder with a joint model that returns logits, then call HaasEngine::compile_vocabulary with the vocabulary and the matching Haas tokenizer.model. Use the compiled vocabulary with transcribe_samples_with_vocabulary, the batch vocabulary APIs, or StreamingSession::windowed_with_vocabulary. For service sessions, supply a VocabularyRequest when starting the source.

CLI

Build the included command-line tool:

cargo build --release --workspace --locked

Download Haas and Silero into a local cache:

./target/release/mozu download-models "$HOME/Library/Caches/mozu-audio"

Append 15s to download the 1,500-frame Haas encoder instead of the default 3,072-frame encoder.

Record and transcribe from the default microphone:

./target/release/mozu record models/haas/assets models/haas/encoder.mlmodelc models/haas/predictor.mlmodelc

Press Enter to start recording and Enter again to stop and transcribe. Enter cancel to discard a recording or q to exit. Microphone access requires permission for the terminal application in macOS settings. Recognition results are written as JSON to standard output.

API constraints

• Token timestamps are decoder positions at 80 ms resolution, rather than word-level forced alignment.
• Cancellation is cooperative; an individual Core ML prediction already in progress completes before cancellation takes effect.