voxr 0.7.0

Audio and speech: codecs, DSP, G2P, voices, and TTS/ASR pipelines over boostr models
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143

//! Load-time and per-request settings for [`VoxCpm2Engine`](super::engine::VoxCpm2Engine).

use numr::dtype::DType;
use std::path::PathBuf;
use std::time::Duration;

/// Generation settings applied to every request. Defaults are the clone
/// pipeline's verified values.
#[derive(Debug, Clone)]
pub struct VoxCpm2SynthOptions {
    /// Flow-matching solver steps per patch, for a request that names
    /// none.
    pub n_timesteps: usize,
    /// Classifier-free guidance scale, for a request that names none.
    pub cfg_value: f32,
    /// Patches during which the stop token is ignored.
    pub min_len: usize,
    /// Base seed for a request that names none. Equal requests at equal
    /// seeds render equal audio on one backend.
    pub seed: u64,
    /// Patches generated between two chunks handed to a streaming sink. Each
    /// chunk costs one suffix decode, which re-decodes the containing
    /// window's left context, so a smaller value lowers latency to the first
    /// sample and raises decoder work per utterance. Applies to
    /// `synthesize_stream` only. `0` is rejected at render time.
    pub stream_chunk_patches: usize,
    /// How long the engine's owner thread keeps taking newly queued requests
    /// into the batch it is about to start, measured from the first request
    /// of that batch.
    ///
    /// The thread blocks on an empty queue, so an idle engine pays this only
    /// once a request has arrived, and the drain ends early when the batch
    /// reaches [`max_batch`](Self::max_batch) or the queue closes. A request
    /// that arrives alone and stays alone therefore starts at most this late
    /// and renders as a batch of one.
    pub batch_window: Duration,
    /// Most requests one batch generates together.
    ///
    /// Rounded DOWN to a width the generation loop runs at (1, 2, 4 or 8) and
    /// never below 1. Requests past the cap stay queued and open the next
    /// batch. Every row of a batch steps on every iteration, a finished row
    /// included, so a wider batch trades per-request latency for throughput.
    pub max_batch: usize,
    /// Whether the owner thread keeps ONE batch open and admits queued
    /// requests into its free slots between steps.
    ///
    /// Off, a batch is built, run to completion and torn down, so a request
    /// arriving while one runs waits for that batch to drain. That wait is
    /// the only thing this option removes.
    ///
    /// On (the default), the batch is persistent. A request arriving
    /// mid-generation prefills on the owner thread and joins on the NEXT
    /// step. The batch steps only the rows seated, rounded up to a width the
    /// Euler graph cache holds, so an idle slot costs no compute and a lone
    /// request steps as a batch of one.
    ///
    /// An unused slot costs no memory either. The row allocation opens at one
    /// and grows only as far as concurrency reaches, so
    /// [`max_batch`](Self::max_batch) is a ceiling, not a reservation. It
    /// never shrinks.
    ///
    /// Each SEATED row holds
    /// [`continuous_capacity`](Self::continuous_capacity) cache positions in
    /// both caches for as long as the engine lives. Turn this off where that
    /// per-row reservation is worth more than mid-flight admission.
    ///
    /// A row renders the same samples either way. Its noise draw follows its
    /// own emitted-patch count and seed, neither of which depends on when it
    /// joined or on what else shares the batch.
    pub continuous_batching: bool,
    /// Cache positions one slot of a persistent batch holds, when
    /// [`continuous_batching`](Self::continuous_batching) is on. Ignored
    /// otherwise.
    ///
    /// A batch that is torn down after every generation sizes its caches to
    /// the rows it actually holds. A persistent one is sized ONCE, before it
    /// knows what it will be asked for, so the budget is fixed here.
    ///
    /// One request spends `t_ref + 2 + text_len` positions on its prefill
    /// (zero-shot spends `text_len`, with no reference prefix) plus
    /// `text_len * 6 + 10` on its patches, and the lane table advances one
    /// position past the last patch. The default covers an ordinary clone
    /// request, not the 4096-patch ceiling one request can reach.
    ///
    /// Raising this is not free. The base and residual caches each hold a K
    /// and a V tensor per layer, every one `max_batch` rows by this many
    /// positions, resident for the batch's whole life. Doubling this value or
    /// [`max_batch`](Self::max_batch) doubles that.
    ///
    /// A request needing more than a slot holds is REFUSED, with an error
    /// naming the positions it needs and the positions a slot has. It is
    /// never queued, truncated or allowed to overrun the cache. The model's
    /// rope table caps this in turn: a position the table does not hold can
    /// be filled and never rotated, so a larger value is clamped to the
    /// table.
    pub continuous_capacity: usize,
}

impl Default for VoxCpm2SynthOptions {
    fn default() -> Self {
        Self {
            n_timesteps: 10,
            cfg_value: 2.0,
            min_len: 2,
            seed: 0,
            stream_chunk_patches: 4,
            batch_window: Duration::from_millis(20),
            max_batch: 8,
            continuous_batching: true,
            continuous_capacity: 1024,
        }
    }
}

/// Everything [`VoxCpm2Engine::load`](super::engine::VoxCpm2Engine::load)
/// needs beyond the checkpoint location: the dtype to cast to, per-request
/// generation defaults, and an optional LoRA adapter to fold into the weights.
#[derive(Debug, Clone, Default)]
pub struct VoxCpm2LoadOptions {
    /// Casts every transformer-stack tensor; `None` keeps the checkpoint's
    /// own dtype (BF16).
    pub dtype: Option<DType>,
    /// Casts every AudioVAE DECODER tensor (conv weights/biases, `Snake`
    /// alphas, sr-cond embeds), independently of `dtype`. `None` keeps the
    /// checkpoint's own F32, verified against PyTorch fixtures at that
    /// dtype.
    ///
    /// The encoder is NOT affected: it always loads and runs at F32. Casting
    /// it changes the latent handed to the transformer stack, which shifts
    /// the stack's conditioning and the whole generation, including where it
    /// stops.
    pub vae_decoder_dtype: Option<DType>,
    /// Per-request generation settings — see [`VoxCpm2SynthOptions`].
    pub synth: VoxCpm2SynthOptions,
    /// A LoRA adapter safetensors file, folded into the model's weights
    /// once, at load time. `None` loads the base model, unchanged.
    ///
    /// The adapter is folded in ONCE: an engine serves ONE adapted model for
    /// its lifetime. There is no per-request adapter switching. Serving
    /// several adapters means loading several engines, one adapter each (a
    /// `--tts-model NAME=DIR` bundle per adapter, in blazr's terms).
    pub adapter: Option<PathBuf>,
}