asry 0.1.0

Sans-I/O word-level forced alignment with WhisperX-equivalent accuracy.

Overview

Sans-I/O cut/batch/whisper/align state machine for speech-to-text indexing pipelines, inspired by WhisperX. Asry itself owns no threads, channels, or runtime — you drive it from a single thread (or wrap blocking calls in your async runtime), feeding samples + VAD and pulling commands the runner answers via sync compute primitives (AsrSource, run_one_alignment).

Quick start

The wav2vec2-base-960h tokenizer ships inside the crate (parsed at build time, no serde_json runtime dep) — only the encoder ONNX and the Whisper ggml checkpoint are BYO. Both files live above crates.io's 10 MB hard limit and cannot be bundled. Fetch them once with the pinned commands below; each one verifies SHA-256 before installing so a republished or truncated upstream surfaces as a hard failure rather than silently altering alignment output.

Whisper ggml model (ggml-large-v3-turbo.bin, ~1.6 GB)

ASRY_WHISPER_MODEL_SHA256="1fc70f774d38eb169993ac391eea357ef47c88757ef72ee5943879b7e8e2bc69"
mkdir -p models
TMP="$(mktemp "${TMPDIR:-/tmp}/ggml-large-v3-turbo.XXXXXXXXXX")"
curl --fail --location \
  --output "$TMP" \
  "https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-large-v3-turbo.bin"
ACTUAL="$(shasum -a 256 "$TMP" | awk '{print $1}')"
if [ "$ACTUAL" != "$ASRY_WHISPER_MODEL_SHA256" ]; then
  echo "SHA-256 mismatch: expected $ASRY_WHISPER_MODEL_SHA256, got $ACTUAL" >&2
  rm -f "$TMP"; exit 1
fi
mv "$TMP" models/ggml-large-v3-turbo.bin

Wav2vec2 alignment encoder (English, ~378 MB)

ASRY_W2V_EN_SHA256="00b7cc69516c1ab63c429e63a2b543e4d42bb77441ec5b98ee935de175b00de1"
TMP="$(mktemp "${TMPDIR:-/tmp}/wav2vec2-base-960h.XXXXXXXXXX")"
curl --fail --location \
  --output "$TMP" \
  "https://huggingface.co/onnx-community/wav2vec2-base-960h-ONNX/resolve/main/onnx/model.onnx"
ACTUAL="$(shasum -a 256 "$TMP" | awk '{print $1}')"
if [ "$ACTUAL" != "$ASRY_W2V_EN_SHA256" ]; then
  echo "SHA-256 mismatch: expected $ASRY_W2V_EN_SHA256, got $ACTUAL" >&2
  rm -f "$TMP"; exit 1
fi
mv "$TMP" models/wav2vec2-base-960h.onnx

(Asry's build.rs can fetch these for you. The two opt-ins are independent: ASRY_FETCH_MODEL=1 fetches the whisper checkpoint, ASRY_FETCH_W2V fetches the wav2vec2 alignment encoders — you do not need the 1.6 GB whisper checkpoint just to align. The script enforces the same SHA-256 pins. Plain cargo build makes no network requests.)

Running the forced-alignment tests

ASRY_FETCH_W2V selects per language, so you only download what you intend to run:

Valid codes: en, ja, zh, ko, es, fr, de, it, pt. A typo is a hard build error, not a silent no-op.

Fetch a language's fixture and its alignment tests simply run — no --ignored, no second step:

export ORT_DYLIB_PATH=/opt/homebrew/lib/libonnxruntime.dylib  # ort is load-dynamic
ASRY_FETCH_W2V=en cargo test --features alignment

Skip the fetch and those same tests are reported as ignored, with the exact command that would enable each one. They are never reported as passed: build.rs emits asry_w2v_<code> only once that language's model and tokenizer are on disk and match their SHA-256 pins, and a test is #[ignore]d unless its own language's cfg is set. Force one to run without its fixture (-- --ignored) and it fails loudly rather than finding some way to report green.

That matters more than it sounds. These tests spent the project's entire history behind an option_env! guard that returned early when the fixture was absent — which was every CI run and every default cargo test. Eleven tests, each claiming to load a 378 MB ONNX encoder, reported ok in 0.00s without opening a file. A test that reports success without executing is worse than no test: it occupies the slot a real gate would fill.

Run an end-to-end alignment

use std::path::Path;
use std::sync::{Arc, atomic::AtomicBool};
use asry::{
  AlignWorkItem, Aligner, AlignerKey, AlignmentFallback,
  AlignmentSetBuilder, AsrChunkContext, AsrSource, EnglishNormalizer,
  Lang, WhisperAsrSource, WhisperContext, WhisperContextParameters,
  run_one_alignment,
  core::{Command, Transcriber, TranscriberConfig},
  ort::session::RunOptions,
};

let aligner = Aligner::from_paths(
  Lang::En,
  Path::new("models/wav2vec2-base-960h.onnx"),
  Path::new("models/wav2vec2-base-960h-tokenizer.json"),
  Box::new(EnglishNormalizer::new()),
)?;
let alignment_set = AlignmentSetBuilder::new()
  .with_fallback(AlignmentFallback::SkipChunk)
  .register(AlignerKey::Lang(Lang::En), aligner)
  .build();

let whisper_ctx = Arc::new(WhisperContext::new_with_params(
  Path::new("models/ggml-large-v3-turbo.bin"),
  WhisperContextParameters::default(),
)?);
let mut asr_source = WhisperAsrSource::new(whisper_ctx)?;

let mut transcriber = Transcriber::new(TranscriberConfig::default());
let abort_flag = Arc::new(AtomicBool::new(false));

while let Some(cmd) = transcriber.poll_command() {
  match cmd {
    Command::Asr { chunk_id, samples, params, .. } => {
      let result = asr_source.run_chunk(AsrChunkContext::new(
        &samples, &params, &abort_flag, chunk_id,
      ))?;
      transcriber.handle_asr(chunk_id, result)?;
    }
    Command::Alignment { chunk_id, samples, sub_segments: _, text, language, runs } => {
      // Sans-I/O OOV resolution: per-run detect + decide.
      // Each run gets its own decisions vec sized + ordered
      // by the events `detect_oov` produces for that run's
      // text + language. Whole-chunk fallback (when `runs`
      // is empty) gets one inner vec.
      // `default_oov_decisions` mirrors the historical
      // behaviour (alphanumeric → wildcard, pronounced
      // symbols → fail-closed); swap for
      // `wildcard_all_decisions` (WhisperX 1:1) or write
      // your own per-run / per-language policy.
      let oov_decisions: Vec<Vec<asry::core::ResolvedOov>> = if runs.is_empty() {
        let events = alignment_set.detect_oov(&text, &language)?;
        vec![asry::core::default_oov_decisions(&events)]
      } else {
        alignment_set.detect_oov_per_run(&runs)?
          .iter()
          .map(|events| asry::core::default_oov_decisions(events))
          .collect()
      };

      let job = AlignWorkItem::from_run_alignment(
        &transcriber, chunk_id, samples, text, language,
        runs, abort_flag.clone(),
        oov_decisions,
      ).expect("chunk in flight");
      // Fresh `RunOptions` per chunk so a watchdog's
      // `terminate()` for chunk N does not poison chunk N+1.
      let run_options = RunOptions::new()?;
      let aligned = run_one_alignment(&alignment_set, &job, &run_options)?;
      transcriber.handle_alignment(chunk_id, aligned)?;
    }
  }
}
while let Some(_event) = transcriber.poll_event() {
  /* Transcript.words() carries word-level alignment */
}
# Ok::<(), Box<dyn std::error::Error>>(())

ORT_DYLIB_PATH overrides the default libonnxruntime lookup if you keep the dylib elsewhere. The alignment feature uses ort in load-dynamic mode — cargo build --features alignment succeeds on a clean toolchain (no system libonnxruntime needed at build time), but you must supply one at run time.

Async users (tokio, smol) wrap WhisperAsrSource::run_chunk and run_one_alignment in spawn_blocking and wire shutdown via their own cancellation tokens flipping abort_flag. Calling the chunk's run_options.terminate() from another thread cancels in-flight ORT inference mid-call; the alignment pipeline polls abort_flag between coarse stages too.

Cargo features

The CTC parity tests run as part of the regular test suite — the OOV policy is per-test runtime data (no Cargo feature):

cargo test --features alignment,bench-internals --test whisperx_unit_parity
# 8/8 — tests 1-6 + 8 use `default_oov_decisions` (asry
# default); test 7 (`4,9` digits-comma WhisperX issue #1372)
# uses `wildcard_all_decisions` to opt into WhisperX 1:1
# behaviour for pronounced symbols.

These tests port WhisperX's tests/test_word_timestamp_interpolation.py 1:1 onto asry's CTC pipeline. The alignment-pipeline lib tests (everything under src/runner/aligner/ + src/runner/alignment_pool/) pin the same algorithmic invariants stage-by-stage; median IoU 0.9955–0.9990 across 854 word pairs vs. WhisperX's recorded outputs (measured during initial calibration; see trellis_beam.rs:305-330).

Audio parity fixtures (~283 MB)

The 14 WAV clips + RTTM speaker annotations asry's end-to-end parity tests reference live out-of-tree at Findit-AI/audio-fixtures so they don't bloat asry's git history. Populate them locally with:

bash scripts/fetch_fixtures.sh

The script shallow-clones the sibling repo and lays files out under tests/parity/fixtures/<name>/{clip_16k.wav,reference.rttm} (the layout existing tests expect). Idempotent + cleans the clone unless ASRY_FIXTURES_KEEP_CLONE=1. CI runs the same script when ASRY_FETCH_FIXTURES=1 is set on the workflow run; default cargo test stays network-free.

License

asry is under the terms of both the MIT license and the Apache License (Version 2.0).

See LICENSE-APACHE, LICENSE-MIT for details.

Copyright (c) 2026 FinDIT studio authors.

Bundled-asset attributions propagate to downstream binaries

asry parses one third-party asset into every compiled binary via include_str! at build time:

The full SPDX expression for an alignment-enabled build is therefore (MIT OR Apache-2.0) AND Apache-2.0. When you redistribute a binary that depends on asry, reproduce the attribution somewhere a recipient can find — for instance, in your application's "About" or third-party-licenses page.

Models you BYO at runtime (Whisper ggml checkpoints, wav2vec2 encoder ONNX, language-specific aligners) carry their own licenses — see the source links above and on each HuggingFace repo. Mirror copies under huggingface.co/FinDIT-Studio re-export upstream weights without modification; the upstream license applies.