euhadra

A programmable voice input framework — ASR, LLM refinement, and OS integration as composable adapters.

euhadra is named after the Japanese land snail genus Euhadra (マイマイ属). ear → cochlea → snail → Euhadra — a chain from hearing to the framework's identity as a Japan-born OSS project.

What it does

euhadra provides an async pipeline that transforms speech into clean, formatted text — without requiring an LLM:

Microphone / WAV
    → ASR (whisper.cpp local)
    → TextFilter (filler removal: um, uh, えーと...)
    → TextProcessor (self-correction, punctuation, capitalization)
    → [LlmRefiner] (seam only — no implementation ships)
    → Output (clipboard / stdout)

Each stage is a Rust trait. Swap any component without touching the rest.

Install

As a library

[dependencies]
euhadra = "0.3"

The default build is deliberately lean — the pipeline runtime plus the rule-based Tier 1/2 text processing, seven direct dependencies, no ML runtime and no system libraries. Opt into the rest:

Feature	Adds	Cost
(default)	Pipeline runtime, filler filters, self-correction, punctuation, ITN	pure Rust
`onnx`	ONNX ASR adapters, BERT punctuation, NER, embeddings, G2P	ONNX Runtime; needs Rust 1.88
`mic`	Microphone capture (`cpal`)	ALSA headers on Linux (`libasound2-dev`)
`vad`	`EarshotVad`, the neural voice activity detector	one pure-Rust crate; needs Rust 1.87
`clipboard`	`ClipboardEmitter` (`arboard`)	—
`cli`	The `euhadra` binary; implies `mic` + `clipboard`	needs Rust 1.85
`testing`	Mock adapters and the WER/CER evaluation harness	—

Microphone capture is behind a feature because cpal links ALSA on Linux, and a consumer who only wants the text-processing tiers should not have to install system packages to compile.

testing is for building test doubles against euhadra's traits, and for running the evaluation harness. It is off by default because neither is library surface; put it under [dev-dependencies] rather than [dependencies].

Stability

This is 0.x. The adapter traits — AsrAdapter, TextFilter, TextProcessor, LlmRefiner, ContextProvider, OutputEmitter — are the part meant to be stable, because implementing one is the reason to depend on this crate. They will still change if a real integration shows they are wrong, and Phase 2's Command and StructuredInput output modes are expected to move LlmRefiner.

Everything around them is fluid: the builder, the concrete adapters, the evaluation harness. Minor versions may break either group until 1.0. Pin an exact version if that matters to you.

As a CLI

cargo install euhadra --features cli

Getting Started

Prerequisites

Rust (1.78+): https://rustup.rs
whisper.cpp: local ASR engine

Build whisper.cpp:

git clone https://github.com/ggerganov/whisper.cpp
cd whisper.cpp
cmake -B build && cmake --build build --config Release
bash models/download-ggml-model.sh base

Build from source

git clone https://github.com/penta2himajin/euhadra
cd euhadra
cargo build --features cli

Transcribe a WAV file

# Raw whisper transcription
cargo run --features cli -- transcribe \
  --file speech.wav \
  --whisper-cli /path/to/whisper.cpp/build/bin/whisper-cli \
  --model /path/to/whisper.cpp/models/ggml-base.bin \
  --language en

Full pipeline (filter + process)

# English: filler removal + self-correction + punctuation
cargo run --features cli -- dictate \
  --file speech.wav \
  --whisper-cli /path/to/whisper-cli \
  --model /path/to/ggml-base.bin \
  --language en

# Japanese: filler removal (えーと, あの, etc.) + ASR artifact cleanup
cargo run --features cli -- dictate \
  --file speech.wav \
  --whisper-cli /path/to/whisper-cli \
  --model /path/to/ggml-base.bin \
  --language ja

Record from microphone

# Record → transcribe → print to stdout
cargo run --features cli -- record \
  --whisper-cli /path/to/whisper-cli \
  --model /path/to/ggml-base.bin \
  --language en

# Record → transcribe → copy to clipboard
cargo run --features cli -- record \
  --whisper-cli /path/to/whisper-cli \
  --model /path/to/ggml-base.bin \
  --language en \
  --clipboard

Press Ctrl+C to stop recording.

Use as a library

use euhadra::prelude::*;
use euhadra::whisper_local::WhisperLocal;

#[tokio::main]
async fn main() {
    // Minimal: ASR + filler filter + self-correction + punctuation.
    // Only .asr() is required; no LLM is involved anywhere below.
    let pipeline = Pipeline::builder()
        .asr(WhisperLocal::new("whisper-cli", "ggml-base.bin").with_language("en"))
        .filter(FillerFilter::for_language(Language::English))
        .processor(SelfCorrectionDetector::new())
        .processor(BasicPunctuationRestorer)
        .emitter(StdoutEmitter)
        .build()
        .unwrap();

    // Load audio and run it through every configured tier
    let audio = euhadra::whisper_local::read_wav("speech.wav".as_ref()).unwrap();
    let result = pipeline.transcribe(&[audio]).await.unwrap();

    // result.raw_text  — original ASR output
    // result.output    — filtered + processed text
}

For Japanese, change the ASR language and the filter language — nothing else:

let pipeline = Pipeline::builder()
    .asr(WhisperLocal::new("whisper-cli", "ggml-base.bin").with_language("ja"))
    .filter(FillerFilter::for_language(Language::Japanese))
    .processor(SelfCorrectionDetector::new())
    .processor(BasicPunctuationRestorer)
    .emitter(ClipboardEmitter::new())   // requires the `clipboard` feature
    .build()
    .unwrap();

FillerFilter::for_language picks the segmentation the script needs: whitespace for English, Spanish and Korean, 、 for Japanese, ， for Chinese. Pairing these by hand is a real hazard — SimpleFillerFilter splits on whitespace, so a Japanese utterance arrives as a single token and one that opens with a filler is removed in full, leaving an empty transcript and no error. Prefer for_language; reach for a concrete filter only when you need to customise its lexicon, and then keep it matched to the language yourself.

Three-tier text processing

euhadra processes ASR output through three independent layers, each optional:

Tier	Component	What it does	LLM?	Size
1	TextFilter	Filler removal (um, uh, えーと)	No	0 MB (rules)
2	TextProcessor	Punctuation, capitalization, self-correction, term dictionary, NER	No	0 MB (rules) or 5-250 MB (ONNX)
3	LlmRefiner	Tone adjustment, context-adaptive rewriting	Yes	Trait only — nothing ships

Tier 1 + 2 alone produce clean, punctuated text without any LLM or network calls.

Languages

Text processing is per-language work — filler lexicons are hand-written, and punctuation and self-correction behave differently per script. euhadra therefore only claims a language it can measure.

Language	Filler filter	ASR baseline (CI)	Filler F1 gold
English	✅	✅	✅ in tree
Japanese	✅	✅	✅ in tree
Chinese	✅	✅	✅ in tree
Korean	✅	✅	✅ in tree
Spanish	✅	✅	⚠️ generated in CI

The gold sets themselves are provisional. Most were drafted by Claude and have not yet been reviewed by native speakers — this covers the filler sets for English, Japanese, Chinese and Korean, and every self-correction set. The one language whose annotations come from human markup (Spanish, via CIEMPIESS) is the one not currently measured. Native-speaker review of the existing five is the single most useful contribution to this project right now; see CONTRIBUTING.md.

Everything else is unmeasured. The pipeline will still run on other languages — ASR is a pluggable adapter and several backends are multilingual — but the Tier 1 and Tier 2 stages have no lexicon for them and no way to tell you when they are wrong, so treat the output as unvalidated.

Spanish is the case worth understanding. Its gold set is not missing; it cannot be shipped. The source corpus (CIEMPIESS Test) is CC-BY-SA-4.0, so committing derived annotations would propagate ShareAlike into this MIT/Apache tree. The generator is checked in and writes to a gitignored cache instead, and only the resulting scores are committed. That posture is deliberate — but the CI wiring that would run it never landed, so in practice Spanish went unverified, and a defect that silently disabled filler removal for punctuated input survived until every language was run by hand.

Term dictionary

Some words come out wrong no matter how good the model is. Say "typwrtr" and a Japanese ASR returns 「タイプライター」 — which is correct, it is what you said. It is just not what you wanted written. No acoustic model will produce a coined spelling nobody trained on; only you can say what you meant.

use euhadra::prelude::*;

let dictionary = TermDictionary::new(
    [TermEntry {
        term: "typwrtr".into(),
        aliases: vec!["タイプライター".into(), "typewriter".into()],
    }],
    MatchPolicy::for_language(Language::Japanese),
)?;

let pipeline = PipelineBuilder::new()
    .asr(/* ... */)
    .processor(dictionary)
    .build()?;

euhadra owns the behaviour, not the dictionary. No bundled term list, no file format, no load(path) — TermEntry derives Deserialize, so your app reads its own settings in its own format and hands over the entries. A term list shipped by euhadra would be an opinion about what you meant; one you supply is a fact about your vocabulary.

It runs as a pipeline stage rather than your own find-and-replace for one reason: ordering. Run it after BasicPunctuationRestorer and it meets text whose sentence starts are already capitalised; after InverseTextNormalizer and the numerals are already rewritten. Only something inside the pipeline can pick where it sits.

MatchPolicy::for_language chooses scope and normalisation together, because both are language knowledge and both fail silently when guessed:

Language	Scope	case	full-width	kana
English / Spanish / Korean	word boundary	✓	✓	—
Japanese	substring	✓	✓	✓
Chinese	substring	✓	✓	—
`MatchPolicy::none()`	substring	—	—	—

The rule: a fold that loses information is not included. Hiragana and katakana spell the same word, so those fold together. 「タイプライタ」 and 「タイプライター」 need not be the same word — a product name and a common noun can differ by exactly that mark — so the long vowel is left alone. Spanish accent stripping would merge año (year) with ano, so it is not offered. Anything missing is covered by writing one more alias; normalisation here saves typing, it does not decide correctness.

Matching is one pass, longest alias first, and replaced text is never rescanned. A match always replaces — there is no confidence score that might quietly decline, and every substitution is reported as a Correction with a codepoint span so you can show it or undo it.

TermDictionary::new reports everything wrong with a dictionary at once, keyed by entry and alias index, so a settings UI can highlight the offending rows rather than making the user fix one error per save.

One known hazard: a two-character alias is allowed, so registering IT will rewrite every it. Length is a poor proxy — Qz is two characters and harmless — and the real predictor is word frequency, which needs per-language data euhadra does not carry.

Voice activity detection

Without it, euhadra hands the ASR adapter whatever it was given. Record for 30 seconds and speak for 5, and 25 seconds of silence reach the model — which is where "Thanks for watching" and 「ご視聴ありがとうございました」 come from. Add a detector and the silence is dropped before the adapter sees it:

use euhadra::prelude::*;
use euhadra::vad::EarshotVad;          // feature = "vad"

let pipeline = PipelineBuilder::new()
    .asr(/* ... */)
    .vad(EarshotVad::new())
    .build()?;

It sits ahead of the ASR adapter rather than inside microphone capture, so WAV input gets the same treatment.

Two backends. EarshotVad (feature vad) runs a 40 KiB neural network embedded in the earshot crate — pure Rust, no ONNX runtime, no model file to fetch, 16 kHz only. EnergyVad needs no dependency at all and works at any rate, but it decides on loudness, so a keyboard opens an utterance and a quiet speaker eventually stops being heard. Prefer the former.

Where the boundaries fall is a separate decision from which frames are speech, and it lives in Segmenter for both backends. Its defaults deliberately lean towards waiting: a 700 ms minimum silence, so a mid-sentence pause is a breath rather than a boundary. The asymmetry is the point — under-segmenting costs latency, while over-segmenting hands the model a fragment, and a model given a fragment answers fluently and wrongly rather than failing.

Incremental output

A detector also gives you utterances, and an utterance is something you can show the speaker before they stop talking:

let mut session = pipeline.session();
let mut partials = std::mem::replace(&mut session.partials, tokio::sync::mpsc::channel(1).1);
tokio::spawn(async move {
    while let Some(p) = partials.recv().await {
        println!("[{:?}] {}", p.start, p.text);
    }
});
let result = session.finish().await?;

This is not streaming ASR. No bundled adapter exposes a streaming API, and re-transcribing a growing prefix was measured and rejected — the text churned 182% (en) / 350% (ja) and warm RTF went to 1.54, slower than real time. What you get instead is one transcript per utterance, which for dictation is the useful granularity anyway.

Does it help? Measured, yes

On the FLEURS en/ja subsets with 5 s of silence added either side of each utterance, using the models euhadra actually ships. Full table and method in docs/benchmarks/vad_delta_wer.md.

condition	en (WER)	Δ	ja (CER)	Δ
clean, no detector	0.0762	—	0.0724	—
padded, no detector	0.1875	+0.1114	0.1211	+0.0487
padded, `SpeechOnly` (default)	0.0762	+0.0000	0.0759	+0.0035

How much silence costs you depends on the decoder. Ask Canary — the en model — to transcribe 10 seconds of silence alone and it returns a runaway repetition (".S. Sometimes it's a long way, …" fifty times over) or a fluent invented paragraph. Ask Parakeet and it returns 「心の声。」. An attention decoder chooses its own output length; a transducer's is bounded by acoustic frames. Both improve with a detector; only one was dangerous without.

Partials are advisory by default. FinalPass decides what the returned transcript is actually computed from:

Policy	Final transcript	ASR passes
`SpeechOnly` (default)	The detected speech, joined, transcribed as one utterance	2
`WholeUtterance`	The recording exactly as captured	2
`JoinSegments`	The concatenated partials	1

SpeechOnly separates the two failure modes: the silence is gone, but the model still sees each utterance whole, so a boundary placed slightly wrong costs a little padding rather than a fragment. WholeUtterance changes the final text by nothing at all — useful for measuring one policy against another. JoinSegments is the cheap one and the only one that inherits segmentation errors in full. Dropping session.partials skips the per-utterance pass entirely, which saves the second ASR run under the first two policies.

The gap between the first and last is not theoretical. Off identical segmentation, en at −45 dBFS scored 0.0855 under SpeechOnly and 0.3940 under JoinSegments — 4.6× worse, because the fragments went to an attention decoder that answered them fluently and wrongly. Use JoinSegments when one ASR pass matters more than the transcript does.

CLI reference

euhadra dictate     Transcribe a WAV file through the full pipeline
  --file <path>       WAV file (16-bit PCM)
  --whisper-cli       Path to whisper-cli binary
  --model             Path to GGML model
  --language          Language hint (en, ja, etc.)
  --no-filter         Skip filler removal
  --no-process        Skip text processing (punctuation, self-correction)

euhadra record      Record from microphone through the full pipeline
  --whisper-cli       Path to whisper-cli binary
  --model             Path to GGML model
  --language          Language hint
  --clipboard         Output to clipboard instead of stdout
  --no-filter         Skip filler removal
  --no-process        Skip text processing

euhadra transcribe  Whisper-only transcription (no pipeline)
  --file <path>       WAV file
  --whisper-cli       Path to whisper-cli binary
  --model             Path to GGML model
  --language          Language hint

ONNX feature (optional)

For higher-quality text processing with ML models (no Python required):

cargo build --features onnx

This enables:

OnnxPunctuationRestorer — CNN-BiLSTM punctuation + capitalization model
WhisperOnnxAdapter — Whisper-large-v3-turbo ASR via ONNX Runtime (encoder + KV-cached decoder loop). Best CER+RTF for Korean on CPU per the #83 backend bench: 1.09% / 0.484 on FLEURS-ko with the q4 quantisation.

Without the onnx feature, euhadra uses rule-based implementations with zero ML dependencies.

Whisper-ONNX setup

# Downloads tokenizer + q4 ONNX bundle (~900 MB) into vendor/whisper_onnx_turbo
scripts/setup_whisper_onnx_turbo.sh

# Use as the ASR stage
cargo run --release --features onnx --example bench_whisper_onnx_ko -- \
    --model-dir vendor/whisper_onnx_turbo \
    --manifest data/fleurs_subset/ko/manifest.tsv \
    --audio-root data/fleurs_subset

From Rust:

use euhadra::whisper_onnx::WhisperOnnxAdapter;

let asr = WhisperOnnxAdapter::load("vendor/whisper_onnx_turbo")?
    .with_language("ko");
// ...then pass `asr` into PipelineBuilder::asr().

Architecture

[euhadra core (Rust)]
    ├── Pipeline runtime (tokio async)
    ├── ASR adapter trait         → WhisperLocal (whisper.cpp), ParakeetAdapter, ParaformerAdapter (zh)
    ├── TextFilter trait          → FillerFilter::for_language → Simple / Japanese / Chinese / Spanish
    ├── TextProcessor trait       → SelfCorrectionDetector, BasicPunctuationRestorer,
    │                                SpokenFormNormalizer, InverseTextNormalizer,
    │                                TermDictionary, PhonemeCorrector, ParagraphSplitter
    ├── LlmRefiner trait          → no implementation (see below)
    ├── ContextProvider trait     → no implementation (see below)
    ├── OutputEmitter trait       → StdoutEmitter, ClipboardEmitter [clipboard]
    ├── VadBackend trait          → EnergyVad, EarshotVad [vad]; Segmenter holds the
    │                                utterance-boundary policy for both
    ├── [mic] Microphone capture  → cpal, cross-platform
    └── [onnx] ONNX backends      → OnnxPunctuationRestorer, and the embedder / G2P
                                    that PhonemeCorrector and ParagraphSplitter
                                    use when available

euhadra is a library, not an application. It ships the traits; native OS integration — accessibility APIs, global hotkeys, on-device LLM bridges — is something a consuming app provides, not something euhadra links in. Microphone capture and clipboard insertion are the exceptions, and only because they turned out to be solvable in cross-platform Rust.

LlmRefiner and ContextProvider are therefore defined but unimplemented. That is deliberate rather than unfinished: Tiers 1 and 2 have ground truth and are gated on WER/CER/F1 in CI, whereas a free-form LLM rewrite has no test that can assert it is correct. euhadra provides the seam; what you plug into it is your opinion, not ours.

Project structure

src/
  lib.rs               — module declarations
  types.rs             — domain types (AudioChunk, AsrResult, ContextSnapshot, etc.)
  traits.rs            — 4 core adapter traits
  filter.rs            — TextFilter trait + English/Japanese filler filters
  processor.rs         — TextProcessor trait + self-correction + punctuation
  pipeline.rs          — PipelineBuilder + async session runtime
  emitters.rs          — ClipboardEmitter (arboard)
  mic.rs               — Microphone capture (cpal)
  whisper_local.rs     — WhisperLocal ASR adapter (whisper.cpp subprocess)
  onnx_processing.rs   — [onnx feature] ONNX-based filters and processors
  mock.rs              — mock implementations for testing
  prelude.rs           — convenience re-exports
  main.rs              — CLI entry point
models/
  euhadra.als          — Alloy formal model
docs/
  spec.md              — full technical specification
  model-upgrade-candidates.md — model survey + backend calibration log
  model-licenses.md    — upstream license summary for bundled weights

Development

cargo test                  # run unit + integration tests
cargo run --features cli -- --help         # CLI usage
cargo build --features onnx # with ONNX inference (requires ort)

Evaluation

Quality is tracked across three layers (full policy in docs/evaluation.md):

Layer	What it measures	How to run	Where it runs
L1 ASR live smoke	FLEURS WER/CER + RTF + ASR/E2E latency	`cargo eval-l1 -- ...`	Every PR (CI: `evaluate-asr`)
L1 layer fast	Tier 1+2 ablation ΔWER + per-layer μ-bench latency	`cargo eval-l1-fast`	Every PR (CI: `evaluate-fast`)
L2 standard + Robust	LibriSpeech / AISHELL-1 / ReazonSpeech WER + MUSAN/RIR SNR sweep	`cargo eval-l2 -- --dataset … --condition …`	Manual / release-time
L3 direct F1 + ablation	Layer-isolated F1 against annotated data; ΔWER on natural-speech fixtures	`cargo eval-l3 -- --task {filler,self-correction,phoneme-correction,ablation} …`	Manual / research

Regression detection lives in docs/benchmarks/ci_baseline*.json — both the WER/CER + latency snapshot and the tolerance policy travel with the file. Two axes:

Relative: +regression% against the committed baseline (catches drift)
Absolute: hard floors tied to user-perceived dictation quality (RTF ≥ 1.0, latency p50 ≥ 1 s, etc.) that don't move with the baseline

Setup scripts (idempotent, skip-if-present):

scripts/setup_whisper.sh                   # whisper.cpp + ggml-tiny models (zh L1)
scripts/setup_canary.sh                    # canary-180m-flash-onnx (en + es L1, ~213 MB INT8)
scripts/setup_parakeet_ja.sh               # parakeet-tdt_ctc-0.6b-ja ONNX (ja L1, ~2.4 GB)
scripts/setup_paraformer_zh.sh             # FunASR Paraformer-large ONNX (zh L1, ~240 MB)
scripts/download_fleurs_subset.py          # L1 FLEURS subset
scripts/download_l2_data.sh <dataset>      # LibriSpeech / AISHELL-1 / MUSAN / RIR
scripts/download_l2_data.py reazonspeech-test
scripts/download_l3_data.sh <dataset>      # CS2W / TED-LIUM 3
scripts/build_l3_natural_fixtures.py manifest --manifest <path>

License

Licensed under either of

Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

at your option.

Contribution

See CONTRIBUTING.md for how to work on euhadra. The most useful contribution right now is native-speaker review of the evaluation gold sets — see the Languages section above for why.

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

euhadra 0.3.0