whisper-nvidia-cli 0.1.0

One-shot agent-native local NVIDIA GPU speech-to-text CLI powered by whisper.cpp
Documentation

Português Brasileiro

whisper-nvidia-cli

crates.io docs.rs license rust

One-shot local NVIDIA speech-to-text that answers an agent in JSON and then disappears.

What is it

  • whisper-nvidia-cli transcribes audio and video on your own NVIDIA GPU through whisper.cpp.
  • The process is born, executes one verb, writes compact JSON on stdout, and dies.
  • Stdout carries data only, so an agent parses it without stripping banners.
  • Stderr carries every log line, warning, and localised hint.
  • The exit code is the machine-readable verdict; the JSON envelope explains it.
  • Nothing leaves the machine except the model download you asked for.

Why

  • You already pay for a GPU that sits idle while a transcription API bills you per minute.
  • Audio you cannot upload is the audio you most need transcribed.
  • An agent that must scrape a progress bar out of stdout cannot be trusted with a batch.
  • A CLI that keeps a daemon alive turns one failure into a stuck host.
  • This binary removes all four problems at once and leaves no process behind.

Superpowers

  • One-shot lifecycle: no daemon, no background worker, no lock left on disk.
  • Agent-native stdout: one JSON object per success, one JSON envelope per failure.
  • Thirteen distinct exit codes so an agent branches without reading prose.
  • Twenty-three published JSON contracts served by the binary and shipped on disk.
  • Payload reduction inside the binary with --select, --filter, --limit, --count-only.
  • Explicit target designation: a side-effect verb never guesses what you meant.
  • VRAM gate that refuses a load it cannot fit instead of crashing mid-decode.
  • SHA-1 verification of every downloaded model against a persisted catalog.
  • HTTPS-only model origin, refused at config time when it is not https://.
  • Full offline mode with --offline, which refuses every network call.
  • Bilingual stderr in en and pt-BR while the JSON contract stays English.

Quick Start

whisper-nvidia-cli doctor
whisper-nvidia-cli models download small
whisper-nvidia-cli transcribe talk.wav
  • Run doctor first; it reports GPU, driver, free VRAM, toolkit, helper, and engine.
  • Run models download small once; the file is verified by SHA-1 before it is trusted.
  • Run transcribe with a named target and read the single JSON object on stdout.

Installation

cargo install whisper-nvidia-cli --features cuda
  • Build with --features cuda to compile the native CUDA engine.
  • Install the CUDA toolkit with nvcc before building; the toolkit gate fails with exit 69.
  • Build with --features cpu only for a machine without a card, at a large speed cost.
  • Build with no feature at all to get the inspection surface without any inference engine.
  • Read engine in the version envelope to know which backend a binary carries.
whisper-nvidia-cli version
  • The engine field is cuda, cpu, stub, or none.
  • A binary reporting none still serves doctor, inspect, models, config, and schema.

Usage

whisper-nvidia-cli transcribe interview.mp3 --language en --model medium --timestamps
whisper-nvidia-cli transcribe --file a.wav --file b.wav --ndjson
cat clip.wav | whisper-nvidia-cli transcribe --stdin
whisper-nvidia-cli transcribe talk.wav --dry-run
whisper-nvidia-cli inspect movie.mkv
  • Pass one positional target, or repeat --file for a batch, or pass --stdin.
  • --use-active reads the opt-in active-target marker and conflicts with the other two.
  • A batch or --ndjson emits one JSON object per file plus a final summary object.
  • --dry-run validates every knob and exits without touching the GPU.
  • --timeout SECONDS arms a deadline that covers the whole invocation, including a stdin read.

Transcription options

  • --language TAG sets the speech language, or auto to let the model decide.
  • --model NAME picks a catalog model; the seed is large-v3.
  • --beam N sets the beam size within 1..=16; the seed is 5.
  • --vad SENSITIVITY takes 0..=1; the seed is 0, which turns the speech cut OFF entirely.
  • --concurrency N runs parallel copies, bounded by 1..=32 and by free VRAM.
  • --device INDEX selects which GPU runs the model.
  • --timestamps emits one segment per span instead of a single block of text.
  • --output PATH also writes the transcript text to that file, leaving the JSON on stdout untouched.
  • --max-memory BYTES caps process RSS and refuses the run rather than exceeding it.
  • --ndjson emits one JSON object per file plus a final summary object.

A larger --vad cuts more aggressively, and cutting is not free: a sensitivity of 0.5 was measured to split ninety-five seconds of speech into 237 chunks, which strips the surrounding context the model decodes against and invites invented sentences. The seed is 0 because keeping the audio whole is the behaviour that transcribes correctly; raise it only when silence dominates the recording.

Global options

  • --lang en|pt-BR overrides the interface locale for one run; JSON keys stay English.
  • --quiet silences stderr, and --verbose raises it, repeating for more detail.
  • --offline refuses every network call, including a model download.
  • --no-input refuses stdin.
  • --config-dir DIR and --cache-dir DIR isolate the config file and the model cache.
  • --print-schema lists every published JSON contract by name.
  • --timeout SECONDS is global, so models download is bounded by it too.

Supported media

  • Audio containers: ogg, mp3, wav, flac.
  • Video containers through the configured helper: mp4, mov, m4v, mkv, webm, avi.
  • Force a decoder with --input-format ogg|mp3|wav|flac when the extension lies.
  • Anything outside the list fails with exit 65.

Payload reduction

whisper-nvidia-cli models list --select name,size_bytes
whisper-nvidia-cli config list --filter key~download
whisper-nvidia-cli models list --count-only
whisper-nvidia-cli transcribe talk.wav --truncate-content 500
  • The cut happens before serialization, so nothing large is built and thrown away.
  • --select accepts a comma-separated list and is also spelled --fields.
  • --filter accepts key=value, key!=value, and key~substring, and repeats with AND.
  • --limit is also spelled -n, and --max-output-bytes drops trailing rows to fit a cap.
  • --sort orders rows by a key, and --dedupe-by drops later rows that repeat one.
  • --truncate-content shortens long strings by character count instead of dropping rows.

Commands

  • The binary serves twenty-one leaf verbs; run commands to get the machine-readable tree.
whisper-nvidia-cli commands
Verb Purpose
transcribe Transcribe a file, stdin, or a batch of --file targets
inspect Measure media without loading a model
models list List the catalog with presence and verification state
models download Fetch a model and verify its SHA-1 digest
models path Print the resolved on-disk path of one model
models remove Delete one model, guarded by --yes
models verify Re-verify one already downloaded model
doctor Diagnose GPU, driver, VRAM, toolkit, helper, engine
config path Print the config file path
config list List every effective key and value
config get Read one key
config set Write one key
config unset Restore one key to the compiled seed
schema List every contract, or emit one with --name
commands Emit the command tree
version Emit product, contract, and engine identity
completions Emit a shell completion script
licenses Emit third-party attribution
locale show Print the effective interface locale
locale set Persist the interface locale
locale clear Restore locale detection

Configuration keys

  • Configuration lives only in the platform config directory, written through config set.
Key Default Notes
model large-v3 Any catalog name
language auto Speech language or auto
device 0 GPU index
timeout 0 Deadline in seconds, 0 disables
timeout-grace 5 Seconds the watchdog waits past timeout so a batch can finish its summary
max-duration 86400 Media ceiling in seconds
beam 5 Beam width, range 1..=16
vad 0.0 Voice activity threshold, range 0..=1
concurrency 1 Parallel copies, range 1..=32
helper empty Absolute path of the media helper
helper-fallback true Retry a hard decode through the helper
max-memory 0 Process RSS cap in bytes, 0 disables
locale empty en or pt-BR
origin-base https://huggingface.co/ggerganov/whisper.cpp/resolve/main Must start with https://
download-retries 2 Retry budget for a model download
retry-max-elapsed 120 Retry wall clock in seconds
download-timeout 600 Per-request HTTP deadline in seconds
download-backoff-ms 200 Base of the exponential backoff between retries, range 0..=60000
http-proxy empty Proxy URL for downloads; empty is a direct connection
hallucination-filter true Whether the caption filter runs at all
hallucination-patterns empty Regex alternation replacing the seeded caption patterns
log-level warn Standing stderr verbosity; -q and -v still win
log-format text text or json
log-timestamps false Whether stderr lines carry a timestamp
vad-frame-ms 30 Energy window of the cut, range 10..=100
vad-rms-min 0.002 RMS floor just above sensitivity zero, range 0..=1
vad-rms-max 0.1 RMS floor at sensitivity 1.0, range 0..=1
resample-taps 31 Taps in the anti-aliasing filter, odd, range 3..=511
io-chunk-bytes 65536 Streaming buffer size, range 4096..=16777216
probe-poll-ms 25 Child-process poll interval, range 1..=1000
helper-args empty Extra helper arguments, comma separated
nvidia-smi-path empty Absolute path of the NVIDIA query tool; empty searches PATH
audio-extensions ogg,mp3,wav,flac Containers the native decoder accepts
video-extensions mp4,mov,m4v,mkv,webm,avi Containers routed to the helper
content-keys text Envelope keys --truncate-content shortens
  • A range violation through config set is a configuration fault and exits 78.
  • The same violation passed as a flag is an argv fault and exits 2.

Exit codes

Code Meaning
0 Success
2 Usage or argv fault
64 No audio input provided
65 Decode failure, unsupported format, video extraction failure
66 Input file not found
69 Helper missing or failed, CUDA toolkit missing, model download failed
70 GPU missing, VRAM insufficient or unknown, memory cap, inference failure, engine not compiled
74 I/O error
78 Configuration fault, unknown model, digest mismatch, insecure origin, duration cap, unknown device, offline with a missing model, missing --yes
124 Timeout exceeded
130 Interrupted
141 Broken pipe
143 Terminated
  • Only a model download failure sets retryable to true in the error envelope.
  • Every error envelope carries code, category, retryable, hint, and correlation_id.

Environment Variables

  • This product reads no environment variable of its own.
  • There is no WHISPER_NVIDIA_CLI_* knob, and none will be added.
  • Every knob is set by a flag, or persisted through config set, and nothing else.
  • CUDA_VISIBLE_DEVICES is ignored, and its presence prints a warning on stderr.
  • Select a card with --device N or with config set device N.
  • Isolate state with --config-dir and --cache-dir instead of exported variables.

Integration Patterns

  • Read INTEGRATIONS.md for the catalog of agents and platforms.
  • Read docs/AGENTS.md for the contract an agent author consumes.
  • Branch on the exit code first, then parse stdout, and only then read stderr.
  • Fetch a contract before parsing an envelope you have not seen.
whisper-nvidia-cli schema --name transcribe
whisper-nvidia-cli --print-schema transcribe talk.wav
  • Pass --quiet in every pipeline so stderr never competes with the payload.
  • Pass --no-input when no human is present, so a stdin read fails instead of blocking.
  • Pass --offline when the host must not open a socket at all.

Performance

  • The first model load of an invocation is the slow one, and it is reported.
  • The cold_load field is true exactly once per invocation, on the first transcription.
  • A batch reuses the loaded model, so the second file is measurably shorter.
  • --concurrency N runs parallel copies, bounded by 1..=32 and by free VRAM.
  • Every extra copy needs the whole VRAM budget of the model again.
  • processing_time_ms in the envelope is the measurement to trend, not a published benchmark.
  • No throughput number is published here because it belongs to your card, not to this binary.

Memory Requirements

Model File size VRAM required
tiny 77691713 bytes 400 MiB
base 147951465 bytes 600 MiB
small 487601967 bytes 1200 MiB
medium 1533763059 bytes 2500 MiB
large-v3 3095033483 bytes 3500 MiB
  • The gate multiplies the VRAM figure by --concurrency before it admits a load.
  • A load that does not fit is refused with exit 70 and never started.
  • doctor reports suggested_model only when free VRAM was actually measured.
  • Unmeasurable free VRAM yields no suggestion, because a guess is indistinguishable from a reading.

Troubleshooting FAQ

The binary exits 70 and says the GPU is missing

  • Attach an NVIDIA GPU and install its driver, then run doctor again.
  • Read the checks array of doctor; each entry names the gate that failed.

The binary exits 69 on toolkit

  • Install the CUDA toolkit so nvcc is on the path, then rebuild with --features cuda.

The binary exits 70 with engine not compiled

  • The build carries no inference engine; reinstall with --features cuda.
  • Confirm with version that engine reports cuda.

A video file exits 65

  • Configure the helper with config set helper /absolute/path and try again.
  • Enable config set helper-fallback true to retry a hard audio decode through the helper.

A model exits 78 with a digest mismatch

  • Delete the file with models remove NAME --yes and download it again.

The transcription exits 70 on VRAM

  • Choose a smaller model with --model small or lower --concurrency.

An agent blocks forever

  • Pass --no-input so a stdin read fails with exit 64 instead of waiting.
  • Pass --timeout SECONDS so the whole invocation carries a deadline and exits 124.

Documentation

Contributing

Security

  • Read SECURITY.md for the responsible disclosure policy.
  • The product sends no telemetry, and the only outbound traffic is a model download you asked for.
  • The model origin must be https://, and config set origin-base refuses anything else.

Changelog

License