gigastt 0.5.0

gigastt turns any machine into a real-time Russian speech recognition server. One binary, one command, state-of-the-art accuracy — everything runs locally.

cargo install gigastt && gigastt serve
# WebSocket: ws://127.0.0.1:9876/ws
# REST API:  http://127.0.0.1:9876/v1/transcribe

Why gigastt?

GigaAM v3 was trained on 700K+ hours of Russian speech. It delivers better accuracy than Whisper-large-v3 on Russian benchmarks while running faster on Apple Silicon and NVIDIA GPUs. WER measured on 993 Golos crowd-sourced samples (4991 words).

Features

• Real-time streaming — partial transcription via WebSocket as you speak
• REST API + SSE — file transcription with instant or streaming response
• Hardware acceleration — CoreML + Neural Engine (macOS), CUDA 12+ (Linux), CPU everywhere
• INT8 quantization — 4x smaller model, 43% faster inference
• Multi-format audio — WAV, M4A/AAC, MP3, OGG/Vorbis, FLAC
• Speaker diarization — identify who said what (optional feature)
• Automatic punctuation — GigaAM v3 model produces punctuated, normalized text
• Auto-download — model fetched from HuggingFace on first run (~850 MB)
• Docker ready — CPU and CUDA images with multi-stage builds
• Hardened — connection limits, frame caps, idle timeouts, sanitized errors

Quick Start

Install & Run

# From crates.io
cargo install gigastt
gigastt serve

# From source
git clone https://github.com/ekhodzitsky/gigastt
cd gigastt
cargo run --release -- serve

The model (~850 MB) downloads automatically on first run.

Docker

# CPU (any platform)
docker build -t gigastt .
docker run -p 9876:9876 gigastt

# CUDA (Linux, requires NVIDIA Container Toolkit)
docker build -f Dockerfile.cuda -t gigastt-cuda .
docker run --gpus all -p 9876:9876 gigastt-cuda

# Model auto-downloads on first run (~850 MB)

Transcribe a File

# CLI
gigastt transcribe recording.wav

# REST API
curl -X POST http://127.0.0.1:9876/v1/transcribe \
  -H "Content-Type: application/octet-stream" \
  --data-binary @recording.wav
# {"text":"Привет, как дела?","words":[],"duration":3.5}

API

WebSocket — Real-time Streaming

Connect to ws://127.0.0.1:9876/ws, send PCM16 audio frames, receive transcription in real time.

Client                            Server
  |                                 |
  |-------- connect --------------> |
  |                                 |
  | <------- ready ----------------- |
  | {type:"ready", version:"1.0"}  |
  |                                 |
  |------- configure (optional) --> |
  | {type:"configure",              |
  |  sample_rate:16000}             |
  |                                 |
  |-------- binary PCM16 --------> |
  |                                 |
  | <------- partial --------------- |
  | {type:"partial", text:"привет"} |
  |                                 |
  | <------- final ----------------- |
  | {type:"final",                  |
  |  text:"Привет, как дела?"}      |

Supported sample rates: 8, 16, 24, 44.1, 48 kHz (default 48 kHz, resampled to 16 kHz internally).

REST API

SSE streaming example:

curl -X POST http://127.0.0.1:9876/v1/transcribe/stream \
  -H "Content-Type: application/octet-stream" \
  --data-binary @recording.wav
# data: {"type":"partial","text":"привет как"}
# data: {"type":"partial","text":"привет как дела"}
# data: {"type":"final","text":"Привет, как дела?"}

Full protocol spec: docs/asyncapi.yaml

Client Libraries

Ready-to-use WebSocket clients in examples/:

# Python
pip install websockets
python examples/python_client.py recording.wav

# JavaScript (Node.js)
npm install ws
node examples/js_client.mjs recording.wav

Performance

Hardware Acceleration

cargo build --release --features coreml   # macOS: CoreML + Neural Engine
cargo build --release --features cuda     # Linux: NVIDIA CUDA 12+
cargo build --release                     # CPU (any platform)

Features are compile-time and mutually exclusive.

INT8 Quantization

Optional quantized encoder: 4x smaller, ~43% faster, 0% WER degradation (verified on 993 Golos samples / 4991 words). Auto-detected at runtime.

When built with --features quantize, INT8 encoder is created automatically on first download or serve — no manual steps needed.

# Automatic (recommended)
cargo install gigastt --features quantize
gigastt serve   # downloads model + auto-quantizes on first run

# Manual
gigastt quantize                     # native Rust quantization
python scripts/quantize.py           # legacy Python alternative

Architecture

                    Audio Input
                   (PCM16, multi-rate)
                        |
                        v
               +-----------------+
               | Mel Spectrogram |  64 bins, FFT=320, hop=160
               +-----------------+
                        |
                        v
            +------------------------+
            |   Conformer Encoder    |  16 layers, 768-dim, 240M params
            |  (ONNX Runtime)        |  CoreML | CUDA | CPU
            +------------------------+
                        |
                        v
            +------------------------+
            | RNN-T Decoder + Joiner |  Stateful: h/c persisted
            |  (ONNX Runtime)        |  across streaming chunks
            +------------------------+
                        |
                        v
            +------------------------+
            |   BPE Tokenizer        |  1025 tokens
            |   + Auto-punctuation   |
            +------------------------+
                        |
                        v
                  Russian Text

CLI Reference

gigastt [OPTIONS] <COMMAND>

Options:
  --log-level <LEVEL>    Log level [default: info]

Commands:
  serve        Start STT server
  download     Download model (~850 MB)
  transcribe   Transcribe audio file (offline)
  quantize     Quantize encoder to INT8 (requires --features quantize)

gigastt serve [OPTIONS]
  --port <PORT>          Listen port [default: 9876]
  --host <HOST>          Bind address [default: 127.0.0.1]
  --model-dir <DIR>      Model directory [default: ~/.gigastt/models]
  --pool-size <N>        Concurrent inference sessions [default: 4]

gigastt download [OPTIONS]
  --model-dir <DIR>      Model directory [default: ~/.gigastt/models]
  --diarization          Also download speaker diarization model (requires --features diarization)

gigastt transcribe [OPTIONS] <FILE>
  --model-dir <DIR>      Model directory [default: ~/.gigastt/models]
  Supports: WAV, M4A, MP3, OGG, FLAC (mono or auto-mixed)

gigastt quantize [OPTIONS]          # requires --features quantize
  --model-dir <DIR>      Model directory [default: ~/.gigastt/models]
  --force                Re-quantize even if INT8 model exists

Model

GigaAM v3 e2e_rnnt by SberDevices:

Requirements

Security

• Binds to 127.0.0.1 only by default (localhost)
• WebSocket frame limit: 512 KB
• Session pool: max 4 concurrent sessions
• Audio buffer cap: 5 s (streaming) / 10 min (file upload)
• Internal errors sanitized — no path or model leakage to clients
• Idle connection timeout: 300 s

Testing

87 unit tests + 24 e2e tests + load & soak tests:

cargo test                           # 87 unit tests (no model needed)
cargo clippy                         # Lint (zero warnings)

# E2E tests (require model, serial to avoid OOM)
cargo run -- download
cargo test --test e2e_rest --test e2e_ws --test e2e_errors --test e2e_shutdown -- --ignored --test-threads=1

# Load & soak (local only)
cargo test --test load_test -- --ignored
cargo test --test soak_test -- --ignored

License

MIT — see LICENSE

Acknowledgments

• GigaAM by SberDevices — the speech recognition model
• onnx-asr by @istupakov — ONNX model export and reference
• ONNX Runtime — inference engine
• ort — Rust bindings for ONNX Runtime