Skip to main content

Crate valle_asr

Crate valle_asr 

Source
Expand description

§valle-asr

Local speech recognition in Rust with selectable model backends and learned word timestamps. The first backend is Qwen3-ASR, paired with Qwen3-ForcedAligner. This repository is independent of valle.

Linux macOS Windows

§Scope

  • Rust 1.99.0, edition 2024; direct crates use the latest stable releases checked on 2026-10-08, with an exact Cargo.lock.
  • Native CPU inference on Windows, Linux and macOS, each on x86_64 and ARM64, through Candle. Python, libtorch, OpenBLAS and a GPU are not required to transcribe audio.
  • Built-in official Qwen3-ASR 0.6B and ForcedAligner 0.6B downloads. Local directories can also load the matching original Qwen3-ASR 1.7B architecture; 1.7B has not yet been included in the validation matrix.
  • Mono/stereo integer/float WAV input, converted to mono 16 kHz with low-pass resampling. Applications can pass PCM directly. Other media can be decoded by the consumer, including Valle’s existing FFmpeg integration.
  • Complete WAV files are decoded and transcribed in bounded chunks, with silence-aware boundaries. Audio and transcript RAM do not grow with duration when using the file API/CLI. Word timestamps retain the original timeline.
  • Pluggable model interface: applications register models and select by ID. Adding a model family does not change the shared transcript schema.

§Run

cargo run --release -- models
cargo run --release -- download --cache-dir models
cargo run --release -- transcribe speech.wav --cache-dir models --language zh

Word timestamps are enabled by default. Use --text-only to run only ASR, and --offline to require previously downloaded, verified weights. To bypass the downloader and use existing official safetensors directories:

cargo run --release -- transcribe speech.wav \
  --model-dir /path/to/Qwen3-ASR-0.6B \
  --aligner-dir /path/to/Qwen3-ForcedAligner-0.6B \
  --language en --output transcript.json

On Windows, use Windows paths and put the command on a single line in PowerShell. No WSL is involved. Additional options include --chunk-seconds (1–30), --max-new-tokens, and --context for domain vocabulary.

Results contain model ID, language, full text, original duration, segments, and per-word {text, start_ms, end_ms}. Milliseconds are absolute source times. For Chinese and Cantonese, a timestamp unit is one Han character, matching the official processor. Word alignment currently supports Chinese, Cantonese, English, French, German, Spanish, Italian, Portuguese and Russian. Japanese and Korean morphological word segmentation is not implemented; those languages can use text-only ASR. Other ASR languages unsupported by the aligner return an explicit error when word timestamps are requested.

§Long files and memory

Submit a complete file with the same transcribe ... --output transcript.json command. There is no ASR file-duration limit. The decoder reads at most the current chunk plus resampling neighbors, mixes channels during reading, and preserves the global resampling phase across boundaries. A 30-second normalized 16 kHz chunk contains 1,920,000 bytes of F32 PCM, regardless of file duration; source-rate buffers, model weights and inference tensors need additional RAM. Input sample rates up to 384 kHz are supported by the file decoder.

Within the last three seconds of a chunk (or the last quarter of a small chunk), the decoder prefers a quiet interval of at least 120 ms. It does not drop or duplicate source samples. Continuous speech without a suitable pause still uses the hard chunk limit; this is not a full VAD or overlap/deduplication system. Exact digital-zero padding is omitted from neural inference while preserving its offset. Other low-volume sounds are not classified as silence.

With word timestamps, ASR runs first and writes interval/text records to a temporary file. Its neural weights are released before the alignment pass re-reads the same WAV. Each neural model loads once per stage. Final segments are emitted during alignment; text-only mode emits them during the first pass. Mapped BF16 weights are converted directly into owned CPU F32 storage without an intermediate BF16 tensor. Each tensor uses a short-lived mapping, so whole checkpoint mappings do not remain resident alongside the final F32 weights. Tokenizers are shared across chunks. Alignment retains the complete attention context but projects only timestamp positions and releases each layer’s temporary KV after use; ASR keeps its decode KV. The JSON writer also spools text on disk so that the existing full text field does not require a growing RAM buffer. Temporary disk usage and processing time grow with duration, and temporary files are removed on success or error. --output replaces the destination only after successful completion.

The current decoder supports standard RIFF WAV, whose 32-bit size fields impose an approximately 4 GiB container limit (about 37 hours for 16 kHz mono PCM16). RF64 and compressed-media file decoding are not implemented here. Consumers can use their own media decoding for other formats.

§Rust API

[dependencies]
valle-asr = { version = "0.2.0", default-features = false, features = ["qwen3", "download"] }

Model weights are downloaded separately and are not embedded in the crate.

use std::{fs::File, io::BufWriter};
use valle_asr::{AsrEngine, JsonTranscriptWriter, TranscribeOptions, models::qwen3::Qwen3};

let mut engine = AsrEngine::new();
engine.register(Qwen3::load(
    "qwen3-asr-0.6b",
    "models/asr",
    Some("models/aligner".into()),
)?)?;
let mut output = JsonTranscriptWriter::new(BufWriter::new(File::create("transcript.json")?))?;
let summary = engine.transcribe_file(
    "qwen3-asr-0.6b",
    "speech.wav",
    &TranscribeOptions::default(),
    &mut |segment| output.write_segment(segment),
)?;
output.finish(&summary)?;

The existing transcribe(&Audio, ...) API remains available for already-loaded PCM and intentionally returns an in-memory Transcript. For long files, use transcribe_file and a sink that persists or displays segments without retaining them. A consumer collecting every callback into a vector will still grow RAM.

Implement AsrModel to register another model family. AsrEngine dispatches to the selected registered instance. Backends receive the same normalized audio and return the same model-independent result types. Qwen is optional: cargo check --no-default-features --lib builds the core API alone.

§Cancellation

Keep a clone of the request’s CancellationToken in the UI or controlling thread, and call cancel() to stop work:

use valle_asr::{CancellationToken, TranscribeOptions};

let stop = CancellationToken::default();
let options = TranscribeOptions {
    cancellation: stop.clone(),
    ..Default::default()
};
// Pass &options to the inference worker; keep stop in the controlling thread.
stop.cancel();
assert!(options.cancellation.is_cancelled());

CancellationToken::from_shared_flag(Arc<AtomicBool>) also accepts a host’s existing stop flag without a monitoring thread. The flag must only transition from false to true during a request. Cancellation is permanent for all token clones; use a fresh token for the next request.

Qwen checks cancellation while decoding/resampling WAV chunks, converting weights, computing mel frames, running encoder/decoder/classifier layers and generating tokens. Both ASR and alignment passes check the flag, so word transcription can stop before its first result callback. Standalone alignment can use Qwen3::align_with_cancellation; align retains its original behavior.

A cancelled request returns an error identifiable with error.is::<valle_asr::Cancelled>(), including when error context is attached; it never reports successful partial output. Already delivered sink chunks cannot be retracted. Call a transactional output writer’s finish() only after successful inference, and reuse the backend with a fresh token after cancellation. Custom backends must check the token during their own compute; the engine also checks before dispatch, around sink callbacks and before returning success.

Cancellation is cooperative: a running tensor/CPU operator or blocking file operation finishes before the next checkpoint. Model constructors and downloads have no cancellation parameter. No fixed wall-clock cancellation latency is promised, and no extra polling thread is created by the library.

§Features

  • qwen3: the built-in Qwen3 CPU backend.
  • download: revision-pinned model downloads with size and SHA-256 checks.
  • cli: the command-line application, including qwen3 and download.

All three are enabled by default. The model-independent core can be used with default-features = false and no additional features.

§Cache behavior

The model cache defaults to VALLE_ASR_CACHE when set, otherwise to the native user cache directory. --cache-dir overrides it. Every revision has its own directory and cross-process writer lock. Existing files are reused only after size and SHA-256 validation; interrupted .part files are never treated as models. Downloads are verified before a final rename. Revisions are immutable while model files are mapped. Do not modify a directory being used for inference.

Tokenizers remain in memory. A bounded resident cache holds one neural model: all ASR chunks run before the weights are released and all alignment chunks run. Repeated text-only transcription and standalone alignment reuse their resident model; full transcription switches models once per stage. This avoids retaining both sets of F32 weights on small CI runners. Qwen3::unload releases the resident model explicitly.

GitHub Actions caches Cargo dependencies/builds per OS and CPU architecture and shares the revision-pinned model cache across platforms. Model cache keys include the hash of models.json; each restored cache is still validated before use. Text files use LF on every platform so Windows checkout does not change content-based cache keys; WAV fixtures remain binary.

§Validation

OSCPURust targetGitHub runner
Linuxx86_64x86_64-unknown-linux-gnuubuntu-24.04
Linuxarm64aarch64-unknown-linux-gnuubuntu-24.04-arm
Windowsx86_64x86_64-pc-windows-msvcwindows-2022
Windowsarm64aarch64-pc-windows-msvcwindows-11-arm
macOSarm64aarch64-apple-darwinmacos-15
macOSx86_64x86_64-apple-darwinmacos-15-intel

Each Linux, macOS and Windows x86_64/ARM64 combination has an independent native workflow, with all six triggered in parallel. CI checks both the runner CPU and Rust host triple before compilation. Each runs formatting, Clippy, the permissive-license gate, optional feature compilation, API/unit tests, then actual 0.6B transcription and word alignment. Missing weights fail the model test.

The real tests check English/Chinese character error rates, non-collapsed monotonic word spans inside source chunks, and offsets on a two-chunk clip. Every platform also runs a two-hour sparse WAV through the streaming API/JSON writer, continuous resampling tests, and real Qwen ASR/alignment on speech at both ends of a two-hour sparse WAV. The long fixture is mostly digital silence; it validates file handling, bounded chunks and global word offsets, not two hours of continuous-speech accuracy or throughput. Each platform uploads the output JSON and a timing/accuracy receipt. They validate correctness on hosted CPUs; performance and Windows hardware compatibility still need real-machine testing. GPU acceleration is not validated in this CPU matrix.

cargo fmt --all -- --check
cargo clippy --locked --all-targets -- -D warnings
python scripts/check_licenses.py
cargo test --locked --release
# Download once, then run actual inference without network access:
cargo run --locked --release -- download --cache-dir models
VALLE_ASR_TEST_CACHE=models cargo test --locked --release --test real_models -- --ignored --nocapture

See THIRD_PARTY.md for exact source revisions, affected code, fixture provenance, preserved notices and dependency licenses. Only permissively licensed references are used. Project license: Apache-2.0.

§Crate development and release checks

The crate targets Rust 1.99+ and edition 2024. Run formatting, Clippy, unit/API tests, documentation and packaging checks before a release. Public APIs must document units, callback execution, resource ownership and failure conditions; missing public documentation and broken Rustdoc links fail validation. The README provides the crate-level documentation, and its Rust example is compiled as a documentation test. Changes to native sources must retain upstream notices, source hashes and reproducible patches. The permissive-license policy in THIRD_PARTY.md applies.

The default features are qwen3, download and cli. Library consumers can use default-features = false with features = ["qwen3"] for local model directories, or add download for the verified cache. No-default-features builds expose the model-independent core. The cli feature enables the backend and downloader; Clap is excluded from builds that omit cli. This ASR backend uses portable Rust CPU inference.

Each independent Windows/Linux/macOS workflow checks supported feature combinations, warning-free API documentation and cargo publish --dry-run, which extracts and builds the exact crate archive without uploading it. Packaging explicitly includes source, model catalogs, test fixtures and third party notices. Model weights, generated audio, local artifacts and build caches are excluded.

cargo fmt --all -- --check
cargo clippy --locked --all-targets --all-features -- -D warnings
cargo test --locked
cargo test --locked --no-default-features
cargo doc --locked --no-default-features --features qwen3,download --no-deps --lib
cargo package --locked --list
cargo publish --locked --dry-run

The current release is version 0.2.0. Breaking public API changes require a minor version increment before 1.0; compatible fixes use a patch increment. Document changes in the release notes and tag each published version. The minimum Rust version is declared in Cargo.toml and matches the toolchain used in CI.

Before publishing, require all six platform/architecture workflows, including the explicit real-model tests, to pass, review the archive’s contents and licenses, and confirm the version is new on crates.io. A dry run does not reserve the crate name or verify the publisher account’s ownership. Publishing a GitHub Release tagged v<version> triggers .github/workflows/release.yml. The tag must match Cargo.toml and belong to main history; all six independent platform/architecture workflows must have passed at that exact commit. The workflow uses the repository secret CARGO_REGISTRY_TOKEN, validates the archive before uploading, and checks its SHA-256 against crates.io. Re-running an identical published archive is safe; an existing version with different contents is rejected. Manual dispatch can validate an existing tag with publish: false before uploading it.

§0.2.0 release notes

  • Native CPU support for Windows, Linux and macOS on both x86_64 and ARM64.
  • Real Qwen3-ASR and ForcedAligner 0.6B validation on all six targets, including English/Chinese recognition, word timestamps, long files and cancellation.
  • Six independent native CI workflows with architecture-specific build caches and shared revision-pinned model weights.
  • Release and permissive-license checks cover all six targets; no intentional Rust API breaking changes in this release.

§0.1.0 release notes

  • Initial Rust library and CLI with pluggable ASR backends.
  • Qwen3-ASR 0.6B/1.7B local loading and Qwen3-ForcedAligner word timestamps.
  • Bounded WAV chunking and streamed JSON for long files.
  • Revision-pinned, verified model downloads and permissively licensed sources.
  • Independent Linux, Windows and macOS CI with real-model and crate checks.
  • Shared host cancellation flags, typed cancellation errors and reusable backends.

Modules§

cache
Revision-pinned, SHA-256-verified model downloads shared by every backend.
models
Built-in optional model backends. Built-in inference backends. Enable qwen3 to use the first model family.

Structs§

AsrEngine
A registry that dispatches requests to independently registered models.
Audio
Validated, normalized mono PCM. Model backends receive 16 kHz f32 samples.
AudioChunk
One normalized source interval. Sample offsets are on the 16 kHz timeline.
CancellationToken
A cloneable, thread-safe stop signal shared by a request and its caller. Cancellation is permanent for every clone. Use a new token for a new request.
Cancelled
A cooperative cancellation returned instead of a successful partial result. Use anyhow::Error::is::<Cancelled>() to recognize it through error context.
JsonTranscriptWriter
Writes the existing Transcript JSON schema without retaining its segments or full text in RAM. A temporary text spool is removed on success or error.
ModelInfo
Model identity and timestamp capabilities reported by a backend.
Segment
Text and word spans for a source interval.
TranscribeOptions
Per-request recognition controls; defaults select 30-second chunks and words.
Transcript
An in-memory transcription; use the file API to bound result memory.
TranscriptSummary
Metadata returned by file transcription; segments are delivered to a sink. It deliberately contains no growing transcript text or segment vector.
WavChunks
A seekable WAV decoder with bounded input/output buffers. No whole-file PCM is retained. Integer/float channels are mixed while decoding, before resampling. Resampling uses absolute positions and real neighboring source samples, so chunk boundaries do not reset its phase or introduce artificial padding.
Word
One aligned word, or one Han character for Chinese/Cantonese.

Enums§

TimestampMode
Select whether recognition also performs learned word alignment.

Traits§

AsrModel
The extension point for model families. No Qwen types appear in this contract.