Expand description
§valle-asr
Local speech recognition in Rust with selectable model backends and learned
word timestamps. The first backend is Qwen3-ASR, paired with
Qwen3-ForcedAligner. This repository is independent of valle.
§Scope
- Rust 1.99.0, edition 2024; direct crates use the latest stable releases
checked on 2026-10-08, with an exact
Cargo.lock. - Native CPU inference on Windows, Linux and macOS, each on x86_64 and ARM64, through Candle. Python, libtorch, OpenBLAS and a GPU are not required to transcribe audio.
- Built-in official Qwen3-ASR 0.6B and ForcedAligner 0.6B downloads. Local directories can also load the matching original Qwen3-ASR 1.7B architecture; 1.7B has not yet been included in the validation matrix.
- Mono/stereo integer/float WAV input, converted to mono 16 kHz with low-pass resampling. Applications can pass PCM directly. Other media can be decoded by the consumer, including Valle’s existing FFmpeg integration.
- Complete WAV files are decoded and transcribed in bounded chunks, with silence-aware boundaries. Audio and transcript RAM do not grow with duration when using the file API/CLI. Word timestamps retain the original timeline.
- Pluggable model interface: applications register models and select by ID. Adding a model family does not change the shared transcript schema.
§Run
cargo run --release -- models
cargo run --release -- download --cache-dir models
cargo run --release -- transcribe speech.wav --cache-dir models --language zhWord timestamps are enabled by default. Use --text-only to run only ASR, and
--offline to require previously downloaded, verified weights. To bypass the
downloader and use existing official safetensors directories:
cargo run --release -- transcribe speech.wav \
--model-dir /path/to/Qwen3-ASR-0.6B \
--aligner-dir /path/to/Qwen3-ForcedAligner-0.6B \
--language en --output transcript.jsonOn Windows, use Windows paths and put the command on a single line in
PowerShell. No WSL is involved. Additional options include --chunk-seconds
(1–30), --max-new-tokens, and --context for domain vocabulary.
Results contain model ID, language, full text, original duration, segments,
and per-word {text, start_ms, end_ms}. Milliseconds are absolute source times.
For Chinese and Cantonese, a timestamp unit is one Han character, matching the
official processor. Word alignment currently supports Chinese, Cantonese,
English, French, German, Spanish, Italian, Portuguese and Russian. Japanese
and Korean morphological word segmentation is not implemented; those
languages can use text-only ASR. Other ASR languages unsupported by the
aligner return an explicit error when word timestamps are requested.
§Long files and memory
Submit a complete file with the same transcribe ... --output transcript.json
command. There is no ASR file-duration limit. The decoder reads at most the
current chunk plus resampling neighbors, mixes channels during reading, and
preserves the global resampling phase across boundaries. A 30-second normalized
16 kHz chunk contains 1,920,000 bytes of F32 PCM, regardless of file duration;
source-rate buffers, model weights and inference tensors need additional RAM.
Input sample rates up to 384 kHz are supported by the file decoder.
Within the last three seconds of a chunk (or the last quarter of a small chunk), the decoder prefers a quiet interval of at least 120 ms. It does not drop or duplicate source samples. Continuous speech without a suitable pause still uses the hard chunk limit; this is not a full VAD or overlap/deduplication system. Exact digital-zero padding is omitted from neural inference while preserving its offset. Other low-volume sounds are not classified as silence.
With word timestamps, ASR runs first and writes interval/text records to a
temporary file. Its neural weights are released before the alignment pass
re-reads the same WAV. Each neural model loads once per stage. Final segments
are emitted during alignment; text-only mode emits them during the first pass.
Mapped BF16 weights are converted directly into owned CPU F32 storage without
an intermediate BF16 tensor. Each tensor uses a short-lived mapping, so whole
checkpoint mappings do not remain resident alongside the final F32 weights.
Tokenizers are shared across chunks. Alignment
retains the complete attention context but projects only timestamp positions
and releases each layer’s temporary KV after use; ASR keeps its decode KV.
The JSON writer also spools text on disk so that the existing full text field
does not require a growing RAM buffer. Temporary disk usage and processing time
grow with duration, and temporary files are removed on success or error.
--output replaces the destination only after successful completion.
The current decoder supports standard RIFF WAV, whose 32-bit size fields impose an approximately 4 GiB container limit (about 37 hours for 16 kHz mono PCM16). RF64 and compressed-media file decoding are not implemented here. Consumers can use their own media decoding for other formats.
§Rust API
[dependencies]
valle-asr = { version = "0.2.0", default-features = false, features = ["qwen3", "download"] }Model weights are downloaded separately and are not embedded in the crate.
use std::{fs::File, io::BufWriter};
use valle_asr::{AsrEngine, JsonTranscriptWriter, TranscribeOptions, models::qwen3::Qwen3};
let mut engine = AsrEngine::new();
engine.register(Qwen3::load(
"qwen3-asr-0.6b",
"models/asr",
Some("models/aligner".into()),
)?)?;
let mut output = JsonTranscriptWriter::new(BufWriter::new(File::create("transcript.json")?))?;
let summary = engine.transcribe_file(
"qwen3-asr-0.6b",
"speech.wav",
&TranscribeOptions::default(),
&mut |segment| output.write_segment(segment),
)?;
output.finish(&summary)?;The existing transcribe(&Audio, ...) API remains available for already-loaded
PCM and intentionally returns an in-memory Transcript. For long files, use
transcribe_file and a sink that persists or displays segments without retaining
them. A consumer collecting every callback into a vector will still grow RAM.
Implement AsrModel to register another model family. AsrEngine dispatches
to the selected registered instance. Backends receive the same normalized
audio and return the same model-independent result types. Qwen is optional:
cargo check --no-default-features --lib builds the core API alone.
§Cancellation
Keep a clone of the request’s CancellationToken in the UI or controlling
thread, and call cancel() to stop work:
use valle_asr::{CancellationToken, TranscribeOptions};
let stop = CancellationToken::default();
let options = TranscribeOptions {
cancellation: stop.clone(),
..Default::default()
};
// Pass &options to the inference worker; keep stop in the controlling thread.
stop.cancel();
assert!(options.cancellation.is_cancelled());CancellationToken::from_shared_flag(Arc<AtomicBool>) also accepts a host’s
existing stop flag without a monitoring thread. The flag must only transition
from false to true during a request. Cancellation is permanent for all token
clones; use a fresh token for the next request.
Qwen checks cancellation while decoding/resampling WAV chunks, converting
weights, computing mel frames, running encoder/decoder/classifier layers and
generating tokens. Both ASR and alignment passes check the flag, so word
transcription can stop before its first result callback. Standalone alignment
can use Qwen3::align_with_cancellation; align retains its original behavior.
A cancelled request returns an error identifiable with
error.is::<valle_asr::Cancelled>(), including when error context is attached;
it never reports successful partial output. Already delivered sink chunks
cannot be retracted. Call a transactional output writer’s finish() only after
successful inference, and reuse the backend with a fresh token after cancellation.
Custom backends must check the token during their own compute; the engine also
checks before dispatch, around sink callbacks and before returning success.
Cancellation is cooperative: a running tensor/CPU operator or blocking file operation finishes before the next checkpoint. Model constructors and downloads have no cancellation parameter. No fixed wall-clock cancellation latency is promised, and no extra polling thread is created by the library.
§Features
qwen3: the built-in Qwen3 CPU backend.download: revision-pinned model downloads with size and SHA-256 checks.cli: the command-line application, includingqwen3anddownload.
All three are enabled by default. The model-independent core can be used with
default-features = false and no additional features.
§Cache behavior
The model cache defaults to VALLE_ASR_CACHE when set, otherwise to the native
user cache directory. --cache-dir overrides it. Every revision has its own
directory and cross-process writer lock. Existing files are reused only after
size and SHA-256 validation; interrupted .part files are never treated as
models. Downloads are verified before a final rename. Revisions are immutable
while model files are mapped. Do not modify a directory being used for inference.
Tokenizers remain in memory. A bounded resident cache holds one neural model:
all ASR chunks run before the weights are released and all alignment chunks
run. Repeated text-only transcription and standalone alignment reuse their
resident model; full transcription switches models once per stage. This
avoids retaining both sets of F32 weights on small CI runners. Qwen3::unload
releases the resident model explicitly.
GitHub Actions caches Cargo dependencies/builds per OS and CPU architecture
and shares the revision-pinned model cache across platforms. Model cache keys include the
hash of models.json; each restored cache is still validated before use.
Text files use LF on every platform so Windows checkout does not change
content-based cache keys; WAV fixtures remain binary.
§Validation
| OS | CPU | Rust target | GitHub runner |
|---|---|---|---|
| Linux | x86_64 | x86_64-unknown-linux-gnu | ubuntu-24.04 |
| Linux | arm64 | aarch64-unknown-linux-gnu | ubuntu-24.04-arm |
| Windows | x86_64 | x86_64-pc-windows-msvc | windows-2022 |
| Windows | arm64 | aarch64-pc-windows-msvc | windows-11-arm |
| macOS | arm64 | aarch64-apple-darwin | macos-15 |
| macOS | x86_64 | x86_64-apple-darwin | macos-15-intel |
Each Linux, macOS and Windows x86_64/ARM64 combination has an independent native workflow, with all six triggered in parallel. CI checks both the runner CPU and Rust host triple before compilation. Each runs formatting, Clippy, the permissive-license gate, optional feature compilation, API/unit tests, then actual 0.6B transcription and word alignment. Missing weights fail the model test.
The real tests check English/Chinese character error rates, non-collapsed monotonic word spans inside source chunks, and offsets on a two-chunk clip. Every platform also runs a two-hour sparse WAV through the streaming API/JSON writer, continuous resampling tests, and real Qwen ASR/alignment on speech at both ends of a two-hour sparse WAV. The long fixture is mostly digital silence; it validates file handling, bounded chunks and global word offsets, not two hours of continuous-speech accuracy or throughput. Each platform uploads the output JSON and a timing/accuracy receipt. They validate correctness on hosted CPUs; performance and Windows hardware compatibility still need real-machine testing. GPU acceleration is not validated in this CPU matrix.
cargo fmt --all -- --check
cargo clippy --locked --all-targets -- -D warnings
python scripts/check_licenses.py
cargo test --locked --release
# Download once, then run actual inference without network access:
cargo run --locked --release -- download --cache-dir models
VALLE_ASR_TEST_CACHE=models cargo test --locked --release --test real_models -- --ignored --nocaptureSee THIRD_PARTY.md for exact source revisions, affected code, fixture provenance, preserved notices and dependency licenses. Only permissively licensed references are used. Project license: Apache-2.0.
§Crate development and release checks
The crate targets Rust 1.99+ and edition 2024. Run formatting, Clippy, unit/API tests, documentation and packaging checks before a release. Public APIs must document units, callback execution, resource ownership and failure conditions; missing public documentation and broken Rustdoc links fail validation. The README provides the crate-level documentation, and its Rust example is compiled as a documentation test. Changes to native sources must retain upstream notices, source hashes and reproducible patches. The permissive-license policy in THIRD_PARTY.md applies.
The default features are qwen3, download and cli. Library consumers can use
default-features = false with features = ["qwen3"] for local model directories,
or add download for the verified cache. No-default-features builds expose the
model-independent core. The cli feature enables the backend and downloader;
Clap is excluded from builds that omit cli. This ASR backend uses portable Rust CPU inference.
Each independent Windows/Linux/macOS workflow checks supported feature
combinations, warning-free API documentation and cargo publish --dry-run,
which extracts and builds the exact crate archive without uploading it.
Packaging explicitly includes source, model catalogs, test fixtures and third
party notices.
Model weights, generated audio, local artifacts and build caches are excluded.
cargo fmt --all -- --check
cargo clippy --locked --all-targets --all-features -- -D warnings
cargo test --locked
cargo test --locked --no-default-features
cargo doc --locked --no-default-features --features qwen3,download --no-deps --lib
cargo package --locked --list
cargo publish --locked --dry-runThe current release is version 0.2.0. Breaking public API changes require a minor
version increment before 1.0; compatible fixes use a patch increment. Document
changes in the release notes and tag each published version. The minimum Rust
version is declared in Cargo.toml and matches the toolchain used in CI.
Before publishing, require all six platform/architecture workflows, including
the explicit real-model tests, to pass, review the archive’s contents and licenses, and confirm
the version is new on crates.io. A dry run does not reserve the crate name or
verify the publisher account’s ownership. Publishing a GitHub Release tagged v<version> triggers
.github/workflows/release.yml. The tag must match Cargo.toml and belong to
main history; all six independent platform/architecture workflows must have
passed at that exact commit. The workflow uses the repository secret
CARGO_REGISTRY_TOKEN, validates the archive before uploading, and checks its
SHA-256 against crates.io. Re-running an identical published archive is safe;
an existing version with different contents is rejected. Manual dispatch can
validate an existing tag with publish: false before uploading it.
§0.2.0 release notes
- Native CPU support for Windows, Linux and macOS on both x86_64 and ARM64.
- Real Qwen3-ASR and ForcedAligner 0.6B validation on all six targets, including English/Chinese recognition, word timestamps, long files and cancellation.
- Six independent native CI workflows with architecture-specific build caches and shared revision-pinned model weights.
- Release and permissive-license checks cover all six targets; no intentional Rust API breaking changes in this release.
§0.1.0 release notes
- Initial Rust library and CLI with pluggable ASR backends.
- Qwen3-ASR 0.6B/1.7B local loading and Qwen3-ForcedAligner word timestamps.
- Bounded WAV chunking and streamed JSON for long files.
- Revision-pinned, verified model downloads and permissively licensed sources.
- Independent Linux, Windows and macOS CI with real-model and crate checks.
- Shared host cancellation flags, typed cancellation errors and reusable backends.
Modules§
- cache
- Revision-pinned, SHA-256-verified model downloads shared by every backend.
- models
- Built-in optional model backends.
Built-in inference backends. Enable
qwen3to use the first model family.
Structs§
- AsrEngine
- A registry that dispatches requests to independently registered models.
- Audio
- Validated, normalized mono PCM. Model backends receive 16 kHz f32 samples.
- Audio
Chunk - One normalized source interval. Sample offsets are on the 16 kHz timeline.
- Cancellation
Token - A cloneable, thread-safe stop signal shared by a request and its caller. Cancellation is permanent for every clone. Use a new token for a new request.
- Cancelled
- A cooperative cancellation returned instead of a successful partial result.
Use
anyhow::Error::is::<Cancelled>()to recognize it through error context. - Json
Transcript Writer - Writes the existing Transcript JSON schema without retaining its segments or full text in RAM. A temporary text spool is removed on success or error.
- Model
Info - Model identity and timestamp capabilities reported by a backend.
- Segment
- Text and word spans for a source interval.
- Transcribe
Options - Per-request recognition controls; defaults select 30-second chunks and words.
- Transcript
- An in-memory transcription; use the file API to bound result memory.
- Transcript
Summary - Metadata returned by file transcription; segments are delivered to a sink. It deliberately contains no growing transcript text or segment vector.
- WavChunks
- A seekable WAV decoder with bounded input/output buffers. No whole-file PCM is retained. Integer/float channels are mixed while decoding, before resampling. Resampling uses absolute positions and real neighboring source samples, so chunk boundaries do not reset its phase or introduce artificial padding.
- Word
- One aligned word, or one Han character for Chinese/Cantonese.
Enums§
- Timestamp
Mode - Select whether recognition also performs learned word alignment.
Traits§
- AsrModel
- The extension point for model families. No Qwen types appear in this contract.