rightkit-qts

Native Qwen3-TTS inference (GGUF/GGML + shared 12 Hz ONNX vocoder), vendored from QTS v0.2.0; see PINNED.md. Rust library name remains qts.

Compatibility

Existing SynthesizeRequest, ModelPaths, Base x-vector/ICL synthesis, streaming, profiling & 0.6B filename resolution retain their public signatures. Existing synthesize() also retains its zero-embedding fallback. Mode APIs are additive, so existing struct literals need no new fields.

Mode API

use qts::{ModeSynthesizeRequest, PresetSpeaker, QtsMode, Qwen3TtsEngine,
          SynthesizeRequest};

let engine = Qwen3TtsEngine::from_model_dir_for_mode(
    "/models/customvoice", QtsMode::CustomVoice,
)?;
let request = ModeSynthesizeRequest {
    synthesis: SynthesizeRequest {
        text: "Welcome!".into(),
        ..Default::default()
    },
    mode: QtsMode::CustomVoice,
    speaker: Some(PresetSpeaker::Vivian),
    instruct: Some("Speak warmly & slowly.".into()),
    ..Default::default()
};
let audio = engine.synthesize_mode(&request)?;
# Ok::<(), qts::Qwen3TtsError>(())

pub enum QtsMode { Base, CustomVoice, VoiceDesign }
pub enum PresetSpeaker {
    Vivian, Serena, Uncle_Fu, Dylan, Eric, Ryan, Aiden, Ono_Anna, Sohee,
}
pub struct ModeSynthesizeRequest<'a> {
    pub synthesis: SynthesizeRequest,
    pub mode: QtsMode,
    pub instruct: Option<String>,
    pub speaker: Option<PresetSpeaker>,
    pub voice_clone_prompt: Option<&'a VoiceClonePromptV2>,
    pub speaker_embedding: Option<&'a [f32]>,
}

Request mode must match loaded checkpoint mode. Presets use codec embeddings at GGUF-provided IDs. Chinese/Auto preset dialect overrides also come from metadata. VoiceDesign omits speaker embedding entirely. Mode synthesis uses existing incremental text/codec prefill (upstream non_streaming_mode=false). Instruction tokens are projected & prepended separately, preserving assistant/codec alignment:

<|im_start|>user\n{instruct}<|im_end|>\n
<|im_start|>assistant\n{text}<|im_end|>\n<|im_start|>assistant\n

Public additions (Result<T> below means Result<T, Qwen3TtsError>):

QtsMode::as_str(self) -> &'static str
PresetSpeaker::ALL: [PresetSpeaker; 9]
PresetSpeaker::as_str(self) -> &'static str
ModeSynthesizeRequest::validate(&self) -> Result<()>

Qwen3TtsEngine::load_mode(paths: ModeModelPaths, mode: QtsMode) -> Result<Self>
Qwen3TtsEngine::from_model_dir_for_mode(dir: impl AsRef<Path>, mode: QtsMode) -> Result<Self>
Qwen3TtsEngine::mode(&self) -> QtsMode
Qwen3TtsEngine::synthesize_mode(&self, req: &ModeSynthesizeRequest<'_>) -> Result<SynthesizeResult>
Qwen3TtsEngine::synthesize_mode_with_profile(&self, req: &ModeSynthesizeRequest<'_>) -> Result<(SynthesizeResult, SynthesisStageTimings)>
Qwen3TtsEngine::synthesize_mode_streaming<S: StreamingSynthesis + Send>(&self, req: &ModeSynthesizeRequest<'_>, sink: &mut S) -> Result<StreamingSynthesizeResult>

TtsTransformer::load_from_ggufs(talker: &GgufFile, predictor: &GgufFile) -> Result<Self>
TtsTransformer::code_predictor_config(&self) -> &CodePredictorConfig
TtsTransformer::preset_speaker_id(&self, speaker: PresetSpeaker) -> Result<i32>
TtsTransformer::preset_speaker_embedding(&self, speaker: PresetSpeaker, thread_count: usize) -> Result<Vec<f32>>
TtsTransformer::prepend_instruction(&self, inputs: &mut PreparedPrefillInputs, tokens: &[i32], thread_count: usize) -> Result<()>
TextTokenizer::encode_instruction_for_tts(&self, instruct: &str) -> Result<Vec<i32>>
GgufFile::get_str(&self, key: &str) -> Option<String>

CodePredictorConfig exposes hidden_size, n_layers, n_attention_heads, n_key_value_heads, intermediate_size, head_dim, rms_norm_eps, rope_theta & vocab_size. Legacy TtsTransformer::config() remains talker configuration. Predictor embeddings remain talker-wide; projection runs before predictor layers for both prefill & every codebook step. Thus 1.7B uses 2048→1024 while 0.6B uses identity with no projection tensors.

Paths & GGUF export contract

pub struct ModeModelPaths {
    pub talker_gguf: PathBuf,
    pub code_predictor_gguf: PathBuf,
    pub tokenizer_gguf: PathBuf,
    pub vocoder_onnx: PathBuf,
}
ModeModelPaths::from_model_dir(dir: impl AsRef<Path>, mode: QtsMode) -> Self

Default split filenames use mode = base | customvoice | voicedesign:

• qwen3-tts-1.7b-{mode}-talker-f16.gguf
• qwen3-tts-1.7b-{mode}-code-predictor-f16.gguf
• qwen3-tts-1.7b-{mode}-tokenizer.gguf
• qwen3-tts-vocoder.onnx (shared across sizes & modes)

Talker/predictor resolution also accepts q8_0, q6_k, q5_k & q4_k, in that preference order after F16. If a bundled QTS file named qwen3-tts-1.7b-{mode}-{quant}.gguf exists, all three GGUF paths use it. Explicit paths can name any compatible export. Filenames describe local export conventions, not verified downloadable distributions; community GGUFs need QTS tensor names & metadata.

Exports retain existing talker.* & code_pred.* tensor names & tokenizer metadata. New contract:

• Talker width: qwen3-tts.talker.embedding_length (also talker.hidden_size under same namespace, legacy qwen3-tts.embedding_length, or output-norm tensor width).
• Predictor metadata: qwen3-tts.code_pred.{embedding_length,attention.key_length, attention.head_count,attention.head_count_kv,feed_forward_length,layer_count, vocab_size,attention.layer_norm_rms_epsilon,rope.freq_base}. Namespace qwen3-tts.code_predictor.* also works; hidden_size & block_count aliases are accepted. Missing width/head counts/FFN width derive from predictor norm/Q/K/FFN tensors; remaining values inherit legacy talker defaults.
• Optional projection: code_pred.small_to_mtp_projection.weight & .bias. Prefixes talker.code_predictor.small_to_mtp_projection & small_to_mtp_projection also work, in predictor or talker file. GGML shapes must be [talker_width, predictor_width] & [predictor_width]; differing widths require both tensors. F32/F16/quantized weights use normal GGML loading.
• Presets: U32 qwen3-tts.talker.spk_id.{lowercase_speaker}. IDs must fit codec vocabulary; absent requested presets fail explicitly. No numeric preset IDs are built into library.
• Optional dialects: string qwen3-tts.talker.spk_is_dialect.{speaker} plus U32 qwen3-tts.talker.codec_language_id.{dialect}. Boolean false means no dialect.
• Optional model identity strings: qwen3-tts.tts_model_type (base, custom_voice, voice_design) & qwen3-tts.tts_model_size (0.6b, 1.7b). Explicit loader mode applies when identity metadata is absent.

Synthetic tests

Unit fixtures use tiny local GGUFs & tensors. Coverage includes independent 2048/1024 metadata, 4→2 projection math/bias, cached/recompute equivalence for split/equal widths, separate-file loading, invalid projection shapes, metadata-derived presets/dialects, exact prompt token strings for every mode, instruction-prefill placement, omitted design speaker & request validation. No model downloads are required.