# Roadmap
## Goal
`timestretch-rs` powers a DJ deck that feels like hardware: tempo control at
resampler latency, ≤ 15 ms pipeline delay on the primary chain, transparent
keylock at DJ ratios, honest latency contracts everywhere, and one engine
whose live output is the same audio the quality benchmarks measure.
The engine rebuild that was this roadmap's original subject is **complete**
(Stages 1–9, July 2026; wide-range Master Tempo, August 2026). Completed
stages and their evidence live in [LEARNINGS.md](LEARNINGS.md); full stage
texts are in git history (tag `v0.10.0` has the last pre-rewrite version).
The quality-closure phase that followed (Stages 10 and 12–19, spawned
by the August 2026 full-code review against Rubber Band) **completed
2026-08-18**: every stage closed with a recorded owner verdict and its
evidence archived in LEARNINGS.md. What remains in this file is the
settled architecture, the binding policies, the deliberately-parked
ideas (Not a Priority Yet), and the un-scheduled 1.0 path.
## Status (2026-08-05)
Shipped and settled: pull-based stage-graph engine (Tape / Keylock /
WideKeylock profiles), SOLA keylock through ±20% at 12.7 ms, wide-range
Master Tempo at 48.6 ms, batch `stretch()` on the same graph with
sample-exact duration and streaming/offline determinism, artifact-first
transient control, `.tsa` analysis container (v0.11.0), musical key
detection, rigid beat grids for quantized material, machine-verified RT
contract (zero-alloc, WCET-gated), Rubber Band reference gate in CI.
Stage 13 (wide-path phase hygiene) completed 2026-08-06 — four confirmed
correctness bugs fixed, null SER 59→139 dB, owner ±50% verdict "significantly
better," Rubber Band gap narrowed (R3 still ahead); archived in LEARNINGS.md.
Stage 15 (DJ-band ride polish) completed 2026-08-07 (PRs #37/#38) — fade-band
clicks gated, seam comb during sustained mild rides −7.1→−4.4 dB with the
mild-motion bounded recenter, ride-quality harnesses in CI, owner mix-in
listen passed; archived in LEARNINGS.md. Its optional items (correlation
reference, strength gating, modulation_hold wiring) remain evidence-gated
ideas, not scheduled work.
Stage 12 (robustness hardening) completed 2026-08-13 (PRs #46/#48 + the
completion PR) — adversarial harness and deck-gesture soak in the CI
quality gates, bounded-drift gate (worst measured 4.5 ms over an
hour-equivalent), no-panic audit clean, weekly re-seeded fuzz campaign;
three real fixes landed on the way; archived in LEARNINGS.md.
**No stages are open. The quality-closure roadmap completed
2026-08-18** with the Stage 10 owner ear session: annotation click
renders confirmed on the beat for the hip-hop rows and teen-spirit
("everything seems pretty spot on"), and the desktop honest
low-confidence display verified on real material. Stage 10 archived in
LEARNINGS.md with the rest.
Stage 19 (direct-ratio wide path) completed 2026-08-14 (PR #60) — the
PV owns the tempo axis for the wide profile as the graph's demand
inverter; the Stage 11 topology is deleted. Blind exit listen (8
conditions, 4 arms, via the new ab-tui): the roboty floor is GONE from
every slowdown ("really nice / good bass / more open" vs the old arm's
"roboty / underwater / artifacts"); at +50% compression the new head
ties Rubber Band (both "slightly roboty") and only the
decorrelation-flattered batch arm escapes — measured sub-bass balance
is at the ideal (0.537 vs batch's 0.58–0.69), so the residual is the
width preference, not a defect. Zero pipeline latency (the analysis
window is source-side lookahead), determinism sample-identical, full
gate suite green; archived in LEARNINGS.md.
Stage 14 (wide-path consolidation) closed 2026-08-13 — the durable
deliverables stand (dead-code/doc sweep, wide determinism harness,
sub-bass fix, M/S machinery), but the owner ±30/±50 blind listen FAILED
against the pre-consolidation batch PV (~7/8), and the three-session
attribution chain that followed cleared resets, M/S, shared code, and
chunking — the live topology itself (varispeed prepass + PV +
post-resampler, the Stage 11 design) is the roboty floor. The
"offline = live" goal is superseded by Stage 19, which unifies both on
the direct-ratio configuration that auditioned clean; archived in
LEARNINGS.md.
Stage 18 (steady-rate splice-cadence stretch) completed 2026-08-13
(PR #56) — SOLA's elastic drift triggers double at steady transposition
inside the primary DJ window (asymmetric band: slowdown force capped at
the write-head headroom; taper released by T=1.15; shipped cadence on
rides). Harmonic-15 purity at −8%: 22.1 → 62.8 dB, asymmetry gone;
blind owner A/Bs: half cadence beat the old build 6/6, and the exit
listen's "robot" vocabulary collapsed to one "very subtle" mention
across 6 conditions. Rubber Band stays ahead on these excerpts — the
residual gap is dominated by the sub-120 Hz pitch-follow scope line
(see Not a Priority Yet), not granulation. Archived in LEARNINGS.md.
Stage 16 (tonal-HF granulation: measure, then decide) completed
2026-08-13 — blind session on the validity-fixed set (12 conditions,
renders from `7f49a50`): granulation IS audible in context (Rubber Band
cleanest 9/12; ours degraded to "roboty" on the worst sustained-tonal
slowdowns, competitive elsewhere), and the corrected-range re-audition
RE-CONFIRMED Stage 7 — the phase-fixed PV was still the
"robotic/underwater/vocoder" arm blind. Verdict recorded, follow-up
scoped as Stage 18; archived in LEARNINGS.md.
Stage 17 (pitch-shift/batch-resampler correctness) completed 2026-08-13
(PRs #42/#47) — batch anti-aliasing (2:1 alias rejection 1.9 → 89.8 dB),
the `pitch_shift` direction inversion found by review and fixed, gates in
CI, owner bright-mix A/B passed (shipped drums "higher quality", the old
path's artifact subtle); archived in LEARNINGS.md.
## Architecture (settled — decisions, not open questions)
- **Stage-graph engine** in `src/engine/`: fixed-block stages
(`process`, `latency_frames()`, `reset()`, `prime()`), fixed per-profile
chains, varispeed head owning demand inversion.
- **Varispeed-first.** source → sinc varispeed (tempo axis, sample-accurate
retargets, no control glide) → per-profile correction.
- **Keylock profile** (primary deck): LinkwitzRiley8 split at 120 Hz; low
band delay-matched, **not** pitch-corrected (pitch follows tempo — the
Stage 2 falsification verdict); high band corrected by elastic-cursor
SOLA. Full keylock through ±20%, release fade 20.5%→35%, 560-frame
(12.7 ms) contract.
- **WideKeylock profile** (opt-in range setting): full-spectrum FFT-2048 /
hop-256 identity-locked PV + post-resampler, artifact-driven per-band
phase resets, 2144-frame (48.6 ms) contract. Profile switch is a
seek-priced rebuild, never a live morph.
- **Artifact-first analysis**: the `PreAnalysisArtifact` drives splice
protection and phase resets; online detection is the fallback.
- **Single engine, both modes**: offline is the same graph with unlimited
lookahead and a guaranteed artifact; streaming-vs-offline agreement is a
determinism property.
- **RT contract**: pull API, no `Result` and no allocation in the audio
path, WCET-gated, honest per-profile latency reporting.
**Evidence caveat (2026-08-05).** Two of these decisions were settled by
listening against a phase vocoder that carried the correctness defects
Stage 13 fixes — the unwrapped-phase blends in particular manufacture
exactly the "phasey" artifact that condemned it:
- *"SOLA carries the entire corrected range"* (Stage 7 verdict: the
small-FFT PV was audibly phasey at every boundary it was placed behind).
- *The accepted wide-rate Rubber Band gap* (Stage 11 verdict: audibly
behind R3, shippable).
Both verdicts stand as shipped behavior. The second was re-baselined at
Stage 13's exit listen (2026-08-06, commit `fb7dcfa`+sidecar rerun): the
fixed PV sounds "significantly better" at ±50% and the gap to R3
narrowed, though R3 remains ahead — the acceptance holds with fresher,
smaller evidence. The first was RE-CONFIRMED on clean evidence at the
Stage 16 blind re-audition (2026-08-13): the phase-fixed small/medium-FFT
PV behind the split was still the "robotic/underwater/vocoder" arm —
"SOLA carries the corrected range" now rests on an uncontaminated
verdict. The un-keylocked low band is **not**
similarly contaminated — its load-bearing justification at DJ ratios is
the ≤ 15 ms latency budget (a bass-resolving FFT cannot fit), which no
review finding touches; Stage 11's full-spectrum result already shows
corrected bass can win when latency permits. **New evidence on file
(2026-08-13, Stage 18 exit listen, blind)**: with the splice granulation
fixed, "bass sometimes sounds out of key" surfaced on 3 of 6 conditions
at ±8% on bass-forward material (msbwy) — the ±1.3-semitone low-band
detune against the corrected highs is audible once it is no longer
masked. The scope line STANDS (the Stage 2 falsification rejected a
vocoder bass; the latency argument is untouched), but any future
re-litigation starts from this evidence — and a time-domain low-band
corrector was never falsified.
## Binding Policies
- **EDM/DJ-first**: quality gates are DJ material at DJ ratios (0.92–1.08
primary, ±20% secondary), streaming path first. The crate's customer is
the author's DJ application; the public API breaks freely pre-1.0.
- **Owner listening is the binding quality gate.** Metrics are regression
tripwires — they have twice failed to predict ear verdicts (LEARNINGS.md,
Stage 11). Every quality-affecting stage ends with a recorded listen.
- **Falsification first**: risky bets get a cheap kill-experiment with a
named fallback before build-out.
- **The accepted scope lines stay accepted until re-litigated with
evidence**: sub-120 Hz follows tempo at DJ ratios; the wide-rate Rubber
Band gap stands, re-baselined smaller at Stage 13 (2026-08-06: R3
still ahead at ±50%, but the fixed PV is "significantly better" than
the Stage 11 renders).
## Principles
- Fix structure instead of stacking corrective heuristics.
- Every stage ends with the desktop app audibly playing the change —
vertical slices, never plumbing-only stages.
- CI stays green throughout; new gates land with the stage that motivates
them.
## Stage Sequence
Stages 10 and 12–19 are complete or closed; evidence in LEARNINGS.md.
### Stage 20 — Bounded Width Treatment (CLOSED 2026-08-19: killed)
Promoted from Not-a-Priority-Yet 2026-08-18 by owner request; killed by
its own kill experiment the next day. Prototype `cf34fad` (env-gated
`TIMESTRETCH_PROTO_WIDTH`, velvet-decorrelated high-passed mid injected
into the side at the M/S decode seam, mono-exact), calibrated by
measurement to the Stage 14 reference levels (+5 dB ≈ Rubber Band,
+15 dB ≈ the preferred batch arm) and blind-listened 2026-08-19
(8 conditions × 5 arms, sealed key, results in
`target/ab/stage20-width/results.json`).
**Verdict — the mechanism dies, and the motivation has faded:**
- At batch-matched +15 dB the injection reads "underwater, smearing,
robotic" in 7 of 8 conditions. Side LEVEL is not the preference
driver: per-channel PV decorrelation is two independently COHERENT
renders, and a diffuse mid-derived injection cannot fake that
character at any gain.
- At +5 dB it only trades minor flaws with the shipped faithful path —
one clear win, no consistent gain.
- The recurring width preference itself did not reproduce post-Stage-19:
the shipped head now ties or beats the `887d854` batch arm in most
conditions (winning MSBWY +50% outright, previously the batch arm's
showcase), so the masking advantage that drove the old preference is
largely gone. Rubber Band was the session's most consistent arm.
The proto code is removed; the shipped paths stay faithful. Any future
width attempt must start from coherent-channel processing (e.g. a true
per-channel blend, at double PV cost), not side injection — and first
re-establish that a preference still exists. Archived in LEARNINGS.md.
### Stage 21 — Corrected Low Band (CLOSED 2026-08-20: achieved)
Promoted from Not-a-Priority-Yet 2026-08-19 by owner request:
re-litigating the sub-120 Hz pitch-follow scope line on the Stage 18
exit-listen evidence ("bass out of key" blind at ±8% on bass-forward
material, no longer masked by granulation). The Stage 2 rejection that
set the scope line was of a VOCODER bass; a time-domain corrector was
never tried.
**Kill question:** does a SOLA-class low-band corrector — ring reader
at the transposition rate, drift repaid in NCC-aligned PERIOD-length
jumps under long raised-cosine crossfades, quiet-moment opportunistic
splicing — put the bass in key at ±8% while keeping the kick's punch
and phase? Or does any correction of the sub band lose to the honest
pitch-follow bass, vocoder or not?
**Prototype:** env-gated `TIMESTRETCH_PROTO_BASSLOCK=1`
(`bass_sola.rs`), replacing the low branch's pitch-follow delay at the
same nominal lag — band alignment and the 12.7 ms latency contract
unchanged; the read cursor wobbles elastically by up to ± one bass
period, the high-band corrector's contract at larger scale. Mechanism
pinned by unit tests (unity = pure delay; ±8% transposition moves a
bass fundamental within 1% with splice steps bounded by the tone's own
slope). Not RT-vetted (full NCC sweep per splice) and not wired to the
extreme-rate fade — those are build-out work, bought only by survival.
**Falsifier:** blind ±8% on bass-forward material (msbwy, cold heart) —
shipped pitch-follow vs bass-locked proto vs Rubber Band (which
corrects its full spectrum, so it is the "bass in key" reference). If
the corrected bass reads worse than the detuned one (wobble, lost
punch, seam artifacts), the scope line stands re-validated against the
stronger falsifier and the finding is recorded.
**Kill-experiment verdict (2026-08-19, blind, 4 conditions × 3 arms,
sealed key, `target/ab/stage21-bass/results.json`): SURVIVED.** The
corrected bass beat pitch-follow in ALL FOUR conditions — the shipped
detuned bass read as the artifact ("dirty, distorted bass", "bassline
sucks, distorted, hum noise", "strange bass hums") while the proto read
"bass hitting well / open, clean, bass ok / good drums". Measured: the
proto's low band lands within ~20 cents of Rubber Band's in-key
reference on the stable-bass track; shipped sits ~130 cents off (the
±8% detune). One residual on msbwy −8%: "kicks smearing a bit" — the
predicted failure mode of the un-wired onset protection, and still
preferred over the detuned arm.
**Build-out (bought by survival):** onset-protected splicing (wire
`ctx.onsets`, protect kick windows like the high-band corrector); an
RT-safe period estimator (budgeted/incremental NCC — the proto's full
sweep is not WCET-viable); extreme-rate fade and live keylock-toggle
wiring (bass must follow the same fades as the high band); A/B matrix
and WCET gates; blind exit listen. The Stage 2 scope line
("un-keylocked low band won") is superseded: that verdict rejected a
VOCODER bass, and the time-domain corrector wins where the vocoder
lost.
**Built out and shipped** (`dd8ca34` + review follow-ups `41f7ae7`):
lockstep splicing on the channel mean, flux-gated onset protection with
directional due-thresholds (early splices on the floor-draining side so
protection is never bypassed by forced splices), budgeted incremental
period sweep (WCET gate unchanged), correction engagement ramp
(pitch-follow below ~±1% keeps the crossover seam rigid — the Stage 15
contract at bass scale; full correction by ±2%), rest recentering
(quiet-gap splice + micro-trim), toggle/fade blend shared with the high
band. Independent adversarial review caught three interlocking HIGH
findings pre-listen (quiet-detector envelope tracking the waveform's
own ripple; an engagement-ramp dead zone classified as rest; corridor
mean riding up to a full period above nominal) — fixed as one unit,
each pinned by a regression test.
**Exit listen (2026-08-20, blind, 8 conditions × 3 arms, ±8/±4 on
msbwy + cold heart, `target/ab/stage21-exit/results.json`): PASSED.**
The detune vocabulary is gone from the corrected arm everywhere; at
±8% on the bass-forward evidence track the old chain read "bassline
hum, bad" while the new chain read "good, clean". Rubber Band remains
the overall reference. Watch items recorded: a very subtle "fuzz /
bitcrush" on the msbwy bassline (quieter than the hum it replaced), a
possible "minor bass wobble" on cold heart +4%, and kick smear on cold
heart −8% present in BOTH our arms (pre-existing, not the bass
corrector — likely high-band or material).
## Not a Priority Yet
- SIMD / architecture-specific acceleration (WCET gates exist to measure
any attempt against; current headroom is comfortable).
- Cross-frame peak tracking / multi-resolution wide path — the RB-class
coherence work. Revisit only if post-Stage-14 listening still shows the
wide gap mattering in practice; Stage 13 already narrowed it (owner
verdict 2026-08-06) and the acceptance stands.
- Desktop UI/UX polish beyond its role as the reference integration.
- Additional presets, wider API surface, convenience wrappers.
- General-purpose (non-EDM) *stretch* quality (analysis generality is
Stage 10; stretch quality gates stay DJ-material).
## Path to 1.0 (decision pending — not scheduled)
Production grade **as a public library** is a separate road, deferred until
the owner decides the crate should take external customers:
- API freeze and semver discipline; rustdoc completeness; README latency
table, RT contract, and artifact workflow as documented guarantees.
- The non-EDM stretch-quality question answered (gated or documented as a
scope boundary — never silently variable).
- Quality sign-off bus factor: a second listener on the structured
checklist; baselines sanity-checked on a second machine class.
- MSRV and platform policy stated in the README.
Stage 12, a prerequisite for this path, completed 2026-08-13.
## Definition of Success
The engine-rebuild definition (≤ 15 ms primary chain, one engine both
modes, zero corrective heuristics, machine-verified RT contract, external
reference evidence in CI, hardware-feel deck) **holds as of the Stage 9
cutover (2026-07-15)** and must keep holding. This roadmap is done when,
in addition:
- Trustworthy beat grids on everything a DJ loads, gated on an annotated
corpus that includes non-EDM and variable-tempo material (Stage 10,
done 2026-08-18 — hip-hop/rock/live/DnB rows, CI floors, owner
ear-verified annotations; the hip-hop beat-PHASE class is a documented
open frontier where the QM reference also scores zero, gated by the
corpus for whenever it is re-attacked).
- The wide-ratio path is free of known correctness defects (Stage 13,
done), its offline and live renders are the same algorithm, and stereo
coherence is gated (Stage 14).
- Riding the fader degrades nothing that holding it steady doesn't
(Stage 15, done — seam and fade gates hold in CI).
- The tonal-HF granulation floor has a recorded listening verdict
(Stage 16, done 2026-08-13 — audible in context; structural response
scoped as Stage 18, falsification-gated).
- No public path resamples without anti-aliasing (Stage 17, done
2026-08-13 — gates in CI, owner A/B passed).
- No panic is reachable from the public API on arbitrary input (Stage 12,
done 2026-08-13 — adversarial harness in CI + audit).