Skip to main content

Module synth

Module synth 

Source
Expand description

The ftts say synthesis path: text in, 24 kHz PCM out.

This module is where the CLI stops describing the pipeline and runs it. It resolves a checkpoint bundle, hydrates the talker and the codec, tokenizes and wraps the text, derives the prompt header, drives TtsEngine::synthesize over the real QwenGenerator, and hands the generated codes to the codec decoder. What comes back is f32 samples; the WAV writing lives in ftts-core::audio and the sink policy in crate::AudioOutput.

§Why the text is prepared before the engine runs

TtsEngine::synthesize owns text preparation, and normally that is where tokenization happens. Here it happens once, up front, and the engine is handed a preparer that returns that exact result. The reason is the cold text embedding: it is [151936, 2048], and materializing it whole to serve a fifteen-token utterance would cost 1.24 GB. The gather needs the token ids, the generator needs the gathered table, and the generator must exist before synthesize is called — so the ids have to be known first. The engine still receives, verbatim, the PreparedText a fresh call would have produced; nothing is skipped, only ordered.

§Speaker conditioning is derived, never invented

An x-vector prompt conditions on a 1,024-wide speaker embedding. A voice source may be either a precomputed raw vector (1,024 little-endian f32, 4,096 bytes) or reference audio decoded through the pinned 24 kHz log-mel front end and ECAPA encoder. Neither path accepts a fabricated vector.

Structs§

LoadedModel
Every weight and table one say needs, hydrated once.
ModelBundle
The model resources ftts say needs, located relative to one model path.
SynthesizedAudio
A completed synthesis: the codes the talker produced and the audio they decode to.

Constants§

SPEAKER_VECTOR_BYTES
Bytes in a speaker-vector file: 1,024 little-endian f32.

Functions§

generation_error
A model-side failure, for callers that need the engine’s own error type.
read_speaker_vector
Read a precomputed 1,024-wide speaker vector.
speaker_from_voice
Derive an x-vector from an enrolled raw vector or a real reference recording.
synthesize
Run one utterance end to end: text, codes, PCM.
write_speaker_vector_new
Write a raw x-vector without replacing an existing enrollment result.