Qwen3 TTS - Rust CLI tools

A Rust implementation of the Qwen3 Text-to-Speech (TTS) model inference. Provides three cross-platform CLI tools suitable for agentic skills for AI agents and bots.

• tts — generate speech from text with named speaker voices
• voice_clone — clone a voice from reference audio
• api_server — OpenAI-compatible HTTP API server

Supports two backends: libtorch (via the tch crate, cross-platform with optional CUDA) and MLX (Apple Silicon native via Metal GPU).

Learn more:

• A Rust implementation / CLI for Qwen3's ASR (Automatic Speech Recognition or Speech-to-Text) models
• An OpenAI compatible API server for audio / speech
• An OpenClaw SKILL for voice generation. Copy and Paste to your lobster to install it

Quick Start

Install binaries, models, and reference audio for your platform:

curl -sSf https://raw.githubusercontent.com/second-state/qwen3_tts_rs/main/install.sh | bash
cd qwen3_tts_rs

The installer detects your OS, CPU, and NVIDIA GPU (if present), then sets up everything in ./qwen3_tts_rs/.

Text-to-Speech

Generate speech with a named speaker using the CustomVoice model:

./tts models/Qwen3-TTS-12Hz-0.6B-CustomVoice "Hello world, this is a test." Vivian english
# Output: output.wav (24 kHz)

Voice Cloning

Clone a voice from reference audio using the Base model (ICL mode):

./voice_clone models/Qwen3-TTS-12Hz-0.6B-Base reference_audio/trump.wav \
  "Hello, this is a voice cloning test." english \
  "Angered and appalled millions of Americans across the political spectrum"
# Output: output_voice_clone.wav (24 kHz)

API Server

Start the OpenAI-compatible API server with the CustomVoice model:

./api_server models/Qwen3-TTS-12Hz-0.6B-CustomVoice --port 8080

Then call the endpoint:

curl -X POST http://localhost:8080/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{"input": "Hello world!", "voice": "alloy"}' \
  -o output.wav

Reference

tts — Text-to-Speech

tts <model_path> [text] [speaker] [language] [instruction]

Output: output.wav (24 kHz, 16-bit PCM)

Available speakers (CustomVoice models): Vivian, Serena, Ryan, Aiden, Uncle_fu, Ono_anna, Sohee, Eric, Dylan

Instruction examples (1.7B CustomVoice only):

• "Speak in an urgent and excited voice"
• "Speak happily and joyfully"
• "Speak slowly and calmly"
• "Speak in a whisper"

voice_clone — Voice Cloning

voice_clone <model_path> <ref_audio> [text] [language] [ref_text]

Output: output_voice_clone.wav (24 kHz, 16-bit PCM)

Preparing reference audio: Must be mono 24 kHz 16-bit WAV. Convert with ffmpeg:

ffmpeg -i input.m4a -ac 1 -ar 24000 -sample_fmt s16 reference.wav

Example:

./voice_clone models/Qwen3-TTS-12Hz-0.6B-Base reference_audio/trump.wav \
  "Hello, this is a voice cloning test." english \
  "Angered and appalled millions of Americans across the political spectrum"

api_server — OpenAI-Compatible API

api_server <model_path> [--host 127.0.0.1] [--port 8080]

Endpoints:

API Request: POST /v1/audio/speech

{
  "input": "Text to synthesize",
  "voice": "alloy",
  "model": "qwen3-tts",
  "response_format": "wav",
  "speed": 1.0,
  "stream": false,
  "language": "english",
  "instructions": "Speak urgently",
  "audio_sample": "<base64-encoded WAV>",
  "audio_sample_text": "Transcript of the reference audio"
}

Voice name mapping (OpenAI → Qwen3):

You can also pass Qwen3 speaker names directly (e.g., "voice": "vivian").

Streaming: When stream: true and response_format: "pcm", the server returns Server-Sent Events with base64-encoded PCM chunks:

data: {"type":"speech.audio.delta","delta":"<base64 PCM>"}
data: {"type":"speech.audio.done"}

Voice cloning via API:

# Encode reference audio as base64
REF_B64=$(base64 < reference.wav)

curl -X POST http://localhost:8080/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d "{
    \"input\": \"Hello from a cloned voice.\",
    \"voice\": \"alloy\",
    \"audio_sample\": \"$REF_B64\",
    \"audio_sample_text\": \"Transcript of the reference audio\"
  }" -o cloned.wav

Build from Source

macOS (MLX backend)

Requires Apple Silicon Mac, Xcode, and CMake.

brew install cmake
git clone https://github.com/second-state/qwen3_tts_rs.git
cd qwen3_tts_rs
git submodule update --init --recursive
cargo build --release --no-default-features --features mlx

Linux (libtorch backend)

1. Download libtorch from libtorch-releases:

# Linux x86_64 (CPU)
curl -LO https://github.com/second-state/libtorch-releases/releases/download/v2.7.1/libtorch-cxx11-abi-x86_64-2.7.1.tar.gz
tar xzf libtorch-cxx11-abi-x86_64-2.7.1.tar.gz

# Linux x86_64 (CUDA 12.6)
curl -LO https://github.com/second-state/libtorch-releases/releases/download/v2.7.1/libtorch-cxx11-abi-x86_64-cuda12.6-2.7.1.tar.gz
tar xzf libtorch-cxx11-abi-x86_64-cuda12.6-2.7.1.tar.gz

# Linux ARM64 (CPU)
curl -LO https://github.com/second-state/libtorch-releases/releases/download/v2.7.1/libtorch-cxx11-abi-aarch64-2.7.1.tar.gz
tar xzf libtorch-cxx11-abi-aarch64-2.7.1.tar.gz

# Linux ARM64 (CUDA 12.6 / Jetson)
curl -LO https://github.com/second-state/libtorch-releases/releases/download/v2.7.1/libtorch-cxx11-abi-aarch64-cuda12.6-2.7.1.tar.gz
tar xzf libtorch-cxx11-abi-aarch64-cuda12.6-2.7.1.tar.gz

2. Set environment and build:

export LIBTORCH=$(pwd)/libtorch
export LIBTORCH_BYPASS_VERSION_CHECK=1

git clone https://github.com/second-state/qwen3_tts_rs.git
cd qwen3_tts_rs
cargo build --release

Alternatively, use pip-installed PyTorch instead of downloading libtorch:

pip install torch==2.7.1
export LIBTORCH_USE_PYTORCH=1
export LD_LIBRARY_PATH=$(python3 -c "import torch; print(torch.__path__[0])")/lib:$LD_LIBRARY_PATH

Download models and generate tokenizer

After building, download models and generate tokenizer.json for each:

pip install huggingface_hub transformers

huggingface-cli download Qwen/Qwen3-TTS-12Hz-0.6B-CustomVoice --local-dir models/Qwen3-TTS-12Hz-0.6B-CustomVoice

python3 -c "
from transformers import AutoTokenizer
for model in ['Qwen3-TTS-12Hz-0.6B-CustomVoice', 'Qwen3-TTS-12Hz-0.6B-Base', 'Qwen3-TTS-12Hz-1.7B-CustomVoice']:
    path = f'models/{model}'
    try:
        tok = AutoTokenizer.from_pretrained(path, trust_remote_code=True)
        tok.backend_tokenizer.save(f'{path}/tokenizer.json')
        print(f'Saved {path}/tokenizer.json')
    except Exception as e:
        print(f'Skipped {model}: {e}')
"

Binaries are in target/release/: tts, voice_clone, api_server.

Rust library usage

Add to your Cargo.toml:

[dependencies]
qwen3-tts-rs = "0.2"
# Or for MLX backend:
# qwen3-tts-rs = { version = "0.2", default-features = false, features = ["mlx"] }

See the API documentation on docs.rs for library usage examples.

Performance (Apple M4 Mac, MLX backend)

Test sentences: ~15–20 words in English ("The quick brown fox..." / "Scientists have discovered...") and Chinese. RTF = Real-Time Factor (wall time / audio duration). Lower is better; < 1.0 means faster than real-time.

0.6B CustomVoice

CLI (tts / voice_clone)

API server (after warmup)

1.7B CustomVoice

CLI (tts)

API server (after warmup)

0.6B vs 1.7B comparison

Architecture

Text → Tokenizer → Dual-stream Embeddings → TalkerModel (28-layer Transformer)
                                                    ↓
                                              codec_head → Code 0
                                                    ↓
                                        CodePredictor (5-layer Transformer) → Codes 1-15
                                                    ↓
                                              Vocoder → 24kHz Waveform

License

Apache-2.0

Credits

Based on the original Python implementation by the Alibaba Qwen team.