Crate ptts 

Phonon: text to 24 kHz speech, on device.

Text is tokenized, a flow-matching language model turns the tokens into Mimi codec latents, and the Mimi decoder turns those into PCM. Generation is streaming throughout: latents are decoded as they are produced.

Getting started

synth::Synth is the whole pipeline behind one call:

use ptts::preprocess::Lang;
use ptts::synth::Synth;
use ptts::checkpoint::read_config;

let tts = Synth::builder(
    read_config("model/config.json")?,
    "model/model.safetensors",
    Lang::En,
)
.tokenizer_file("model/tokenizer.json")
.add_voice("default", "model/default-voice.safetensors")
.build()?;
let pcm = tts.say("Hello world")?;
ptts::wav::write_wav_file("out.wav", &pcm, tts.sample_rate())?;

Layers

Module	Role
synth	The one-call API: load, prime, generate, decode. Start here.
checkpoint	Resolving a model: a local directory, an hf:// repo, or the default.
voice_catalog	Voices listed in a catalog, downloaded when first used.
loader	Reading weights and voice files, and the checkpoint key mapping.
error	Error, what those two return.
plan	Frame and KV budgets, the end-of-speech policy.
preprocess	Per-language text normalization, applied before tokenizing.
tok	The Hugging Face tokenizer, behind the hf feature.
audio	Decoding and resampling audio files for voice cloning, behind audio.
tts_model	tts_model::TTSModel, the streaming primitives synth drives.
flow_lm, transformer	The token-conditioned flow-matching LM.
mimi	The neural audio codec.

Errors

synth and loader return Error; the model modules below them keep xn::Result, because a shape mismatch inside the codec is not something a caller acts on. ? crosses the boundary in both directions, so a frontend whose own functions return xn::Result keeps compiling. The variants are failure classes, so a binding maps them onto its host language’s exceptions in one match.

checkpoint resolves a model for native callers: a local directory, or with the hub feature an hf://owner/name repo on the Hugging Face Hub, which is the only time the library downloads anything. Frontends load checkpoint::DEFAULT_MODEL when none is given. The lower-level builder still accepts explicit files.

Text is normalized before it is tokenized — see preprocess. Which language, or preprocess::Normalize::OFF, is a required argument to synth::SynthBuilder::new: the model reads normalized text noticeably better, but the spoken forms are per-language, so guessing is worse than doing nothing.

A voice is conditioned on once per synth::Synth, whichever entry point is used; synth::SynthApi::session additionally pins the KV budget for a stream of requests.

Callers that need to drive the loop themselves — a browser build stepping from an event loop, with no threads to spawn — should use tts_model::TTSModel directly. Synth is a composition of those primitives, not a replacement for them.

Re-exports

pub use error::Error;

pub use error::Result;

Modules

audioaudio

Reading audio files, for voice cloning.

checkpointNon-WebAssembly

Shared native checkpoint file resolution.

comp

A small Rust DSP library: an AudioModule trait, a Chain that runs modules in series, and a few mono f32 modules — Gain, Compressor, Limiter. Chains can be built in code or loaded from TOML.

error

The error type the API layer returns.

flow_lm

The flow-matching language model. Conditioned on the voice and the text tokens, it emits one Mimi latent per frame, and with it the logit that decides when speech ends.

hubNon-WebAssembly

Checkpoints on the Hugging Face Hub, named hf://owner/name or hf://owner/name@revision.

loader

Reading checkpoints and voice embeddings off disk.

mimi

The Mimi neural audio codec.

plan

Generation policy: how many frames to run, how much KV budget to reserve, and when to stop.

preprocess

Text normalization applied before tokenization.

s2s

Speech to speech: a Phonon checkpoint conditioned on the input audio’s own Mimi latents, one per frame, instead of text, emitting the output latents audio_delay frames behind.

synthNon-WebAssembly

One-call speech synthesis.

tokhf

A crate::Tokenizer backed by Hugging Face tokenizers.

transformer

The streaming transformer both the flow LM and Mimi are built on, with its KV caches.

tts_model

TTSModel, the flow LM and the Mimi decoder together, and TTSConfig, a checkpoint’s config.json.

utils

Loudness normalization for voice prompts: normalize_loudness brings a recording to a fixed level before the speaker encoder embeds it.

voice_catalogNon-WebAssembly

Voice catalogs: voices listed in a JSONL file, each downloaded only when it is used.

wav

Writing PCM as a 16-bit WAV file.

Traits

Tokenizer