rlx-orpheus

Orpheus TTS on RLX — Canopy Orpheus speech LM (Llama-3B GGUF) + SNAC 24 kHz neural codec decoder.

Quick start

just fetch-orpheus fetch-orpheus-snac
just export-orpheus-snac
export ORPHEUS_SNAC_PATH=/tmp/rlx-weights/snac/snac_24khz_decoder.safetensors

just orpheus-demo                    # Metal LM + eager SNAC → /tmp/orpheus-demo.wav
just orpheus-coreml-demo             # Metal LM + native RLX Ane SNAC (macOS, --features coreml)

Architecture

CoreML is native RLX (no ONNX). The RVQ lookup stays on the host; only the conv decoder stack is compiled per latent-length bucket and cached (ORPHEUS_SNAC_AOT).

Cargo features

Build examples:

cargo build -p rlx-orpheus --release --features apple-silicon   # Metal LM + CoreML SNAC
cargo build -p rlx-orpheus --release --features all-backends    # every LM backend + coreml

Audio samples

Bundled MP4 clips (mono AAC, 24 kHz) under assets/:

Regenerate:

just export-orpheus-audio-assets
# optional: refresh sentence MP4s from new synthesis WAVs
export ORPHEUS_SAMPLE_LONG_WAV=/tmp/orpheus-long.wav
export ORPHEUS_SAMPLE_LONGER_WAV=/tmp/orpheus-longer.wav
just export-orpheus-audio-assets

Requires ffmpeg on PATH. Hi clips only need exported SNAC weights; sentence clips need WAVs from just orpheus-demo (or any synthesis run).

Weights

just fetch-orpheus              # GGUF → /tmp/rlx-weights/orpheus/
just fetch-orpheus-snac         # PyTorch checkpoint → /tmp/rlx-weights/snac/
just export-orpheus-snac        # safetensors + config JSON (+ parity fixtures)
just export-orpheus-snac -- --parity-layers   # also ref_stage_*.npy for layer tests
export ORPHEUS_SNAC_PATH=/tmp/rlx-weights/snac/snac_24khz_decoder.safetensors

CLI download (hf-download feature): --download, --download-orpheus [--quant Q4_K_M], --download-snac.

Manual SNAC export:

python3 -m venv .venv-orpheus && source .venv-orpheus/bin/activate
pip install snac torch safetensors
python3 scripts/export_snac_decoder.py --out /tmp/rlx-weights/snac

CLI

just orpheus -- \
  --weights /tmp/rlx-weights/orpheus/orpheus-3b-0.1-ft-Q4_K_M.gguf \
  --text "Hello from RLX Orpheus." \
  --voice tara \
  --device metal \
  --out /tmp/orpheus.wav

CoreML needs --features coreml at build time:

cargo run -p rlx-orpheus --release --features "llama,coreml,metal" -- \
  --weights /tmp/rlx-weights/orpheus/orpheus-3b-0.1-ft-Q4_K_M.gguf \
  --device coreml \
  --text "Hello from RLX on CoreML." \
  --voice tara \
  --out /tmp/orpheus-coreml.wav

Built-in voices: tara, leah, jess, leo, dan, mia, zac, zoe.

Sampling defaults: temperature 0.6, top-p 0.95, repetition penalty 1.1, stop token 128258 (<custom_token_2> / END_OF_SPEECH). Override via --temperature, --top-p, --greedy, etc.

Voice cloning (pretrained checkpoint)

Finetune-prod GGUF (just fetch-orpheus) supports named voices only. Zero-shot clone needs a pretrained GGUF plus a reference JSON from SNAC encoding.

just orpheus-encode-ref /path/ref.wav "exact spoken transcript"
just orpheus-voice-clone -- \
  --weights /path/to/orpheus-3b-0.1-pretrained.Q4_K_M.gguf \
  --target-text "Hello from a cloned voice." \
  --out /tmp/orpheus_clone.wav

Convert canopylabs/orpheus-3b-0.1-pretrained to GGUF before use. Emotive tags: <laugh>, <chuckle>, <sigh>, … (upstream Orpheus docs).

Library

use rlx_orpheus::{
    BackboneLoadOptions, GenerationConfig, OrpheusTts, resolve_orpheus_device,
};

let runtime = resolve_orpheus_device("metal")?;
let backbone_opts = BackboneLoadOptions::for_device(runtime.lm);
let mut tts = OrpheusTts::load_on_with(
    "/path/to/orpheus.gguf".as_ref(),
    "/path/to/snac_24khz_decoder.safetensors".as_ref(),
    runtime,
    backbone_opts,
)?;
tts.config = GenerationConfig { max_new_tokens: 1200 };
let out = tts.synthesize("Hello world.", Some("tara"))?;
// out.samples — mono f32 PCM at 24 kHz

SNAC-only decode (no LM):

use rlx_orpheus::{SnacBackend, SnacLoadOptions, decode_orpheus_codes};

let snac = SnacBackend::open(path, SnacLoadOptions { coreml: false })?;
let pcm = decode_orpheus_codes(&snac, &frame_codes)?;

// CoreML SNAC path (feature `coreml`, macOS):
let snac = SnacBackend::open(path, SnacLoadOptions { coreml: true })?;

Voice clone API: [VoiceCloneReference::load_json] + [OrpheusTts::synthesize_voice_clone].

Streaming: [OrpheusTts::synthesize_stream] — ~2048-sample PCM chunks after 4 SNAC frames; tail appended from full decode.

Just recipes

Tests

# SNAC decoder vs Python reference (needs ref_noise_*.npy under ORPHEUS_SNAC_REF_DIR)
ORPHEUS_SNAC_PATH=/tmp/rlx-weights/snac/snac_24khz_decoder.safetensors \
ORPHEUS_SNAC_REF_DIR=/tmp/rlx-weights/snac \
cargo test -p rlx-orpheus --features codec --lib decode_matches_reference_when_weights_available --release

# Native CoreML SNAC vs eager (macOS, feature coreml)
ORPHEUS_SNAC_PATH=/tmp/rlx-weights/snac/snac_24khz_decoder.safetensors \
ORPHEUS_SNAC_REF_DIR=/tmp/rlx-weights/snac \
cargo test -p rlx-orpheus --features coreml --test coreml_snac_parity --release

just fetch-orpheus-snac fetch-whisper
just test-orpheus-whisper

# Per-backend LM + Whisper (includes wgpu when built with --features gpu)
just test-orpheus-backends-whisper named_voice_whisper_wgpu

Environment

Notes

• Memory: default [BackboneLoadOptions::for_device] uses GPU prefill on Metal / CUDA / ROCm without eager F32 drain. wgpu/Vulkan use CPU GGUF parity. Avoid ORPHEUS_PACKED_LM=1 (O(n²) recompute). ORPHEUS_HIGH_MEM=1 raises compile cap to 256.
• wgpu / Vulkan: --device gpu selects the portable GPU backend; the 3B GGUF LM runs CPU prefill + CPU decode (GGUF parity, same sampling as Metal). SNAC stays CPU eager. Native wgpu LM decode is pending upstream KV parity (ORPHEUS_MASK_LOGITS=1 only needed for that path).
• MLX: not selected by --device auto; set ORPHEUS_MLX_KV=1 to enable KV decode on MLX.
• Batch decode: [decode_orpheus_codes] returns the full SNAC waveform; streaming uses 4-frame windows (2048-sample centers) and appends the remaining tail automatically.
• Multi-utterance: reuse one [OrpheusTts] session and call [OrpheusTts::synthesize_batch] (same backbone load as [examples/batch_demos.rs]).
• Parity: SNAC z_q + PCM vs Python within ~1e-3 when noise fixtures exist; per-decoder-stage parity with export_snac_decoder.py --parity-layers; CoreML SNAC vs eager within ~0.08 max abs diff (fp16 on ANE). LM: Metal for_tts greedy tokens match CPU synthesis reference (metal_prefill_parity tests).
• Serving / throughput: Orpheus is autoregressive (one SNAC slot token per LM step) — unlike chat LLMs there is no prefill/decode asymmetry to exploit for continuous batching today. rlx-serve continuous + fused batching targets Qwen3Generator::forward_batch_decode; Llama32Generator has no batched decode yet, so multi-request TTS throughput is one utterance per backbone lock until batched Llama decode lands. Reuse [OrpheusTts::synthesize_batch] or a job queue for now.
• Tensor parallel: rlx-distributed implements in-graph TP (Megatron MLP + head-sharded attention) for Qwen3 blocks on the research path — not wired for Llama-3B / Orpheus. A 3B Orpheus model fits one Apple Silicon or consumer GPU; TP would help multi-node serving once Llama blocks expose the same collectives (see distributed-inference-design.md).