ft8coder 0.5.0

CLI tool for encoding FT8 messages to channel symbols
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
# wsjtr

wsjtr is a standalone FT8 decoding library written in Rust. Its algorithms are derived from the reference implementation in WSJT-X by Joe Taylor (K1JT) and the WSJT Development Group, which is licensed under the GNU General Public License v3.0. This project is likewise licensed under GPLv3.

wsjtr can be used as:

* The decoding engine for FT-Activ8, a mobile FT4/FT8 app for Android
* A supplemental decoder for WSJT-X, injecting additional decodes that WSJT-X may have missed
* A tool for exploring how modifications to the FT4/FT8 decoding process affect speed, computational efficiency, and decoding performance.
* A library in your own amateur radio projects.

This project is written and maintained by Brian Bodiya (KC1WIH).

## Usage

### Run wsjtr-supplement (TUI supplemental decoder)

A Windows/Linux terminal UI (ratatui) application that runs `wsjtr` alongside WSJT-X and displays supplemental decodes — messages found by wsjtr but not WSJT-X. Rust replacement for `supplement_wsjtx.py`. Validates unique-to-wsjtr callsigns via QRZ, highlights invalid ones, and can optionally inject supplemental decodes back into WSJT-X.

```bash
target/release/wsjtr-supplement
```

Options:
- `--config <path>`: path to config file (default: `~/.config/wsjtr-supplement/config.toml`)

Configuration (TOML) includes: operator callsign, QRZ credentials, wsjtr binary path and arguments, WSJT-X UDP address/port, early decode delay, and decode injection toggle. Settings can also be edited in-app.

WSJT-X must be configured to send UDP messages (Settings → Reporting → UDP Server). If multicast is not used **only one application will be able to connect to WSJT-X at a time**.

### Run supplement_wsjtx.py (WSJT-X supplemental decoder)

A Linux-only PyQt5 GUI that runs `wsjtr` alongside WSJT-X and displays supplemental decodes — messages found by wsjtr but not WSJT-X. Validates unique-to-wsjtr callsigns via QRZ and highlights invalid ones. This script was the predecessor to wsjtr-supplement.

```bash
python3 supplement_wsjtx.py
```

WSJT-X must be configured to send UDP messages (Settings → Reporting → UDP Server). If multicast is not used **only one application will be able to connect to WSJT-X at a time**.

### wsjtr (multi-pass)

Decode an existing WAV using multi-pass subtraction (per-pass WAVs go into `wsjtr-wav/` by default):

```bash
target/release/wsjtr --wav ../stored-runs/10m_220315/10m_220315.wav --passes 4 --depths 1,2,3,3 --keep-wav
```

Decode multiple WAVs in sequence (enables cross-sequence A7 decoding using callsigns from prior windows):

```bash
target/release/wsjtr --wav file1.wav --wav file2.wav --wav file3.wav -p 3 -d 1,1,3
```

Live decode from audio input:

```bash
target/release/wsjtr --audio-device <device_name> --passes 4 --depths 1,2,3,3 --keep-wav
```

#### Audio backends

wsjtr has two audio capture backends:

| Backend | Platforms | Default on | `--audio-device` format |
|---------|-----------|------------|------------------------|
| **parec** | Linux (PulseAudio/PipeWire) | Linux | PulseAudio source name |
| **miniaudio** | Linux, Windows, macOS | Windows, macOS | Device name substring match |

On Linux, parec is the default because PipeWire/PulseAudio manages audio devices and can route specific sources. Use `--miniaudio` to force the miniaudio backend on Linux. On Windows, miniaudio uses the WinMM backend which sees USB Audio Class 1.0 devices (e.g., Icom IC-7300) that WASAPI may not enumerate.

#### Finding your audio device (Linux — parec, default)

The `--audio-device` flag takes a PulseAudio source name. To list available sources:

```bash
pactl list sources short
```

This prints lines like:

```bash
48  alsa_output.pci-0000_00_1f.3.analog-stereo.monitor  PipeWire  s32le 2ch 48000Hz  IDLE
49  alsa_input.pci-0000_00_1f.3.analog-stereo            PipeWire  s32le 2ch 48000Hz  SUSPENDED
52  alsa_input.usb-Burr-Brown_from_TI_USB_Audio_CODEC-00.analog-stereo  PipeWire  s16le 2ch 48000Hz  RUNNING
```

The second column is the source name to pass to `--audio-device`. Choose the source corresponding to your radio's USB audio interface — typically an `alsa_input.usb-*` entry. Sources named `*.monitor` are loopback monitors of output sinks (speakers), not physical inputs.

If your radio is connected via a USB sound card (e.g., the SignaLink or an Icom's built-in USB audio), look for the USB device name in the source list. You can also use `pactl list sources` (without `short`) for more detail including the `device.description` field, which shows a human-readable name.

To verify you have the right source, record a few seconds and check for audio:

```bash
parec --device=alsa_input.usb-Burr-Brown_from_TI_USB_Audio_CODEC-00.analog-stereo \
  --format=s16le --rate=12000 --channels=1 --file-format=wav test_capture.wav
# Ctrl-C after a few seconds, then play it back or inspect in Audacity
```

#### Finding your audio device (Windows/macOS — miniaudio)

List available input devices:

```bash
target/release/wsjtr --list-devices
```

Pass a substring of the device name to `--audio-device`:

```bash
target/release/wsjtr --audio-device "USB Audio" -p 3
```

If only one capture device is connected (e.g., your radio's USB audio), it will be auto-selected without needing `--audio-device`.

Disable cross-sequence decoding (A7 is enabled by default):

```bash
target/release/wsjtr --audio-device <device_name> -p 3 --no-a7
```

Decode diversity — vary subtraction order for better weak-signal recovery:

```bash
target/release/wsjtr --wav file.wav -p 1 -d 3 -c 2000 -m 100 --no-early-exit \
  --subtraction-order snr_asc
```

Run multiple diversity trials (unions results from N independent decode runs with varied parameters):

```bash
target/release/wsjtr --wav file.wav -p 1 -d 3 -c 2000 -m 100 --no-early-exit \
  --diversity 4 --diversity-strategy candidate-sort --subtraction-order snr_asc
```

If you want to experiment with subtraction timing (debug/compat), tweak the offset applied to decoded `DT`:

```bash
target/release/wsjtr --wav ../stored-runs/10m_220315/10m_220315.wav --passes 10 --depths 3 --no-early-exit --subtract-dt-offset 0
```

### ft8coder

Encode a message to 79 channel symbols:

```
target/release/ft8coder "CQ KC1WIH FN42"
```

### gen_ft8wav

Generate a WAV file from an FT8 message using `gen_ft8wav` (built as part of the `jt9r` crate):

```bash
target/release/gen_ft8wav "CQ KC1WIH FN42" -f 1500 --pad -o cq_KC1WIH.wav
```

Options:
- `-f, --freq <Hz>`: audio frequency offset (default: 1000)
- `-r, --sample-rate <Hz>`: output sample rate (default: 12000)
- `-o, --output <path>`: output WAV file (default: ft8_tx.wav)
- `-p, --pad`: zero-pad to 15 seconds (required for decoding with jt9r/wsjtr)

Roundtrip example — encode a message to WAV, then decode it back:

```
target/release/gen_ft8wav "CQ KC1WIH FN42" -f 1500 --pad -o test.wav
target/release/jt9r test.wav
```

### jt9r

Decode a WAV file (FT8):

```bash
target/release/jt9r ../stored-runs/10m_220315/10m_220315.wav
```

Notes:

- Output format matches `jt9` style: `HHMMSS SNR DT FREQ ~ MESSAGE`
- `SNR` is a WSJT-X-style dB estimate (computed from decoded tones + spectrum baseline, similar to `jt9 --ft8`).
- Internals are modeled after WSJT-X FT8 decoding (downsample + fine sync + soft metrics + hybrid LDPC BP/OSD + subtraction between passes).
- The WSJT-X-style subtraction passes make jt9r heavier than the earlier single-pass scaffold; in `--release` it's typically comparable to `/usr/bin/jt9` on the stored WAVs.

Useful flags:

- `-d, --depth {1,2,3}`: controls how aggressively jt9r searches/decodes.
  - `1`: BP-only LDPC; stricter sync-quality gate; 2 internal subtraction passes
  - `2`: enables OSD fallback; 3 internal subtraction passes
  - `3`: enables OSD fallback plus CQ/MyCall a-priori attempts; 3 internal subtraction passes
- `-s, --sync-min <float>`: candidate sync threshold (used during `sync8`-style candidate selection).
- `-c, --max-candidates <N>` / `-m, --max-decodes <N>`: bounds on work and output.
- `-i, --iterations <N>`: LDPC BP iterations.
- `-f/--freq-min` and `-F/--freq-max`: decoded audio passband (defaults cover the same general range as `jt9 --ft8`).
- `--my-call <callsign>`: operator callsign for AP mode 2 (MyCall) decoding at depth=3.
- `-v`: verbose timings/counters.

### force_decode

Debug tool that bypasses the normal candidate search and forces FT8 decoding at specific frequency and time-offset coordinates. Useful for diagnosing why the sync finder missed a signal, or for testing decoder sensitivity at known signal locations.

```bash
target/release/force_decode ../stored-runs/10m_144115/10m_144130_final.wav \
  --cand 2232,-0.7
```

Options:
- `--cand <freq_hz,dt_s>`: frequency (Hz) and time offset (seconds) to decode at (repeatable for multiple candidates)
- `-d, --depth <1-3>`: decoding depth (default: 3)
- `-i, --iterations <N>`: max LDPC BP iterations (default: 30)


## Development

### Layout

```
wsjtr/
  Cargo.toml
  supplement_wsjtx.py               # WSJT-X supplemental decoder UI
  docs/
    ft8coder.md                     # ft8core/ft8coder implementation reference
    jt9r.md                         # jt9r decoder implementation reference
    wsjtr.md                        # wsjtr multi-pass decoder reference
    cross_sequence_decoding.md      # A7 cross-sequence decoding design
  decoding-experiments/
    wsjtr_wsjtx_capture.py          # Capture wsjtr vs WSJT-X for comparison
    compare_wsjtr_wsjtx_capture.py  # Analyze captured comparison data
  crates/
    ft8core/
      src/
    ft8coder/
      src/
      tests/
    jt9r/
      src/
    wsjtr/
      src/
    wsjtr-supplement/
      src/
    ft8-engine/
      src/
```

### Prerequisites

- Rust toolchain (cargo + rustc)
- `libasound2-dev` (Linux only, required by the miniaudio audio backend)
- `/usr/bin/ft8code` (for end-to-end encoder comparisons)
- `/usr/bin/jt9` (optional, for jt9r output comparisons)

If you just installed Rust using rustup, make sure your shell environment is updated:

```bash
source ~/.cargo/env
```

### Build

From this directory:

```bash
cargo build --release
```

### Cross-compilation (Windows)

Build Windows executables from Linux using the MinGW cross-compiler:

```bash
# One-time setup
sudo apt install gcc-mingw-w64-x86-64
rustup target add x86_64-pc-windows-gnu

# Build
cargo build --release --target x86_64-pc-windows-gnu
```

Output binaries:
```
target/x86_64-pc-windows-gnu/release/force_decode.exe
target/x86_64-pc-windows-gnu/release/ft8coder.exe
target/x86_64-pc-windows-gnu/release/gen_ft8wav.exe
target/x86_64-pc-windows-gnu/release/jt9r.exe
target/x86_64-pc-windows-gnu/release/wsjtr.exe
target/x86_64-pc-windows-gnu/release/wsjtr-supplement.exe

```

Note: On Windows, `wsjtr` uses the miniaudio backend (WinMM) for live capture. Use `--list-devices` to find available input devices, or `--wav` for WAV file decoding.

## Experimentation

### Capture wsjtr vs WSJT-X UDP decodes

To run `wsjtr` and record WSJT-X decodes via its UDP interface at the same time, use:

```bash
python3 decoding-experiments/wsjtr_wsjtx_capture.py --windows 10 --audio-device <device_name>
```

This writes a run folder under `stored-runs/` containing:
- `wsjtr` stdout/stderr and parsed decodes
- WSJT-X UDP messages (including decodes)
- `wsjtr` per-window/per-pass WAV snapshots

WSJT-X must be configured to send UDP messages to the capture tool (Settings → Reporting → UDP Server).

## Benchmarks

Quick timing comparison on one WAV (prints times to stderr):

```bash
cargo build -p jt9r --release

/usr/bin/time -p target/release/jt9r ../stored-runs/10m_220315/10m_220315.wav >/dev/null
/usr/bin/time -p /usr/bin/jt9 --ft8 ../stored-runs/10m_220315/10m_220315.wav >/dev/null
```

## Tests

Run all tests:

```bash
cargo test
```

End-to-end tests compare `ft8coder` output with `/usr/bin/ft8code`. If `/usr/bin/ft8code` is missing, those tests are skipped.

For jt9r comparison against `/usr/bin/jt9`:

```bash
cargo test -p jt9r --test jt9_compare
```

## Notes

- `ft8core` implements the encoder pipeline: pack77, CRC-14, LDPC (174,91), Gray mapping, Costas sync insertion, and runtime callsign hash table.
- `jt9r` uses WSJT-X-style FT8 pieces:
  - `sync8`-style coarse candidate search (time/frequency grid)
  - `sync8d`-style coherent fine sync at 200 Hz complex baseband
  - `ft8b`-style soft metrics (nsym=1/2/3) feeding the LDPC decoder
  - `subtractft8`-style waveform subtraction between internal passes
  - `ft8_a7`-style cross-sequence decoding using callsigns from prior windows
- `wsjtr` maintains a callsign hash table across windows, resolving non-standard callsigns (displayed as `<CALL>`) that were previously seen in unhashed form.
- The design docs (`docs/ft8coder.md`, `docs/jt9r.md`, `docs/wsjtr.md`) describe intended architecture and planned work.

## Architectural Differences from WSJT-X

wsjtr is a from-scratch Rust reimplementation of the FT8 decoding pipeline. While the core DSP algorithms (sync detection, downsampling, soft metrics, LDPC decoding, signal subtraction) are faithful ports of WSJT-X's Fortran routines, the system architecture differs in several significant ways. The WSJT-X reference source used for comparison is 3.0.0-rc1.

### Process model

WSJT-X uses a two-process architecture: a Qt GUI that captures audio and manages state, and a separate `jt9` process that does the actual decoding. They communicate through **Qt shared memory** — a large structure (`dec_data`) containing raw audio samples (180k int16), symbol spectra, and ~90 decoding parameters. The GUI writes audio and sets `ipc(2)=1`; jt9 polls for this flag, decodes, and clears it. This IPC design dates back to when multiple mode decoders (JT9, JT65, FT8, etc.) all lived in the same jt9 binary.

wsjtr is a single binary. In live mode it captures audio on a background thread — using either vendored `miniaudio` (cross-platform via WinMM/CoreAudio/ALSA, default on Windows/macOS) or a `parec` subprocess (PulseAudio/PipeWire, default on Linux) — while the main thread runs the decode loop. There is no shared memory — decoded results are printed directly to stdout. The `jt9r` crate exposes a library API (`Decoder::decode()` / `Decoder::decode_single_pass()`) that wsjtr calls directly, so multi-pass orchestration is just Rust function calls rather than IPC.

### Multi-pass subtraction architecture

Both systems use multi-pass decoding with signal subtraction, but the orchestration differs:

**WSJT-X** runs a single-level loop inside `ft8_decode.f90`. For each of 1–3 passes, it calls `sync8()` to find candidates, then iterates over them calling `ft8b()` to decode. Each successful decode immediately triggers `subtractft8()` to remove that signal from the shared audio buffer `dd`. The next candidate in the same pass sees the already-modified audio. Between passes, the sync threshold is lowered (2.1 → 1.3) to pick up weaker signals revealed by subtraction.

**wsjtr** has a two-level structure:
- **Inner passes** (inside `jt9r`'s `Decoder::decode()`): 2–3 internal passes with subtraction, similar to WSJT-X. This is what you get when running `jt9r` standalone.
- **Outer passes** (inside `wsjtr`'s main loop): Each outer pass re-applies *all* previously decoded signals' subtractions from the original audio, then runs `jt9r::decode_single_pass()` on the residual. This "subtract from scratch" approach avoids accumulating numerical artifacts from sequential subtraction.

The outer passes also support per-pass depth configuration (`--depths 1,2,3,3`), so early passes can use fast BP-only decoding while later passes enable OSD and AP modes on the already-cleaned residual.

### Subtraction algorithm

The core subtraction math is the same — both implement the conjugate-multiply, low-pass filter, reconstruct, subtract approach from the `subtractft8.f90` algorithm:

```
camp(i) = received(i) × conj(reference(i))     # demodulate
cfilt    = LPF(camp)                             # extract amplitude envelope
dd(i)   -= 2 × Re(cfilt(i) × reference(i))      # subtract reconstructed signal
```

Where they differ:

- **DT refinement**: wsjtr optionally searches multiple time offsets around the decoded DT (controlled by `--subtract-dt-range` and `--subtract-dt-steps`), evaluates subtraction quality at each, and uses parabolic interpolation to find the optimal sub-sample offset. This compensates for the ±2.5 ms quantization in the decoded DT. WSJT-X has a simpler `lrefinedt` option that searches ±90 samples but without the parabolic interpolation or quality-based skip logic.
- **Skip logic**: If wsjtr's DT refinement can't find a clear minimum, it skips the subtraction entirely to avoid corrupting the audio. WSJT-X always subtracts.
- **SNR filtering**: wsjtr supports `--subtract-min-snr` to skip subtraction of very weak signals whose decode parameters may be unreliable.

### Early decoding

WSJT-X can attempt decoding before the full 15-second T/R period ends. The `jt9` program triggers `multimode_decoder()` at three symbol counts — 41, 47, and 50 symbols (roughly 9.6s, 11.0s, 11.7s into the period). Earlier passes zero-pad the remaining audio. This lets strong signals appear in the GUI up to 5 seconds before the period ends.

wsjtr implements the same concept at the outer-pass level: configurable per-pass durations (`--durations`) default to 9.6s, 11.0s, and 11.7s. In live mode it waits for sufficient audio before each pass; in offline mode it truncates/zero-pads the WAV accordingly. The difference is that wsjtr's early decoding is part of the multi-pass loop rather than multiple invocations from the GUI.

### Cross-sequence (A7) decoding

wsjtr implements cross-sequence decoding where callsigns decoded in one T/R sequence are used as a priori information in the next. After all passes complete for a window, wsjtr generates candidate messages from previously-seen callsigns (e.g., if KC1WIH was decoded calling CQ in the even sequence, try "KC1WIH W2XYZ -12" combinations in the odd sequence). These candidates are correlated against the residual audio using a distance metric with acceptance thresholds.

WSJT-X 3.0-rc1 has an `ft8_a7` module but its integration is more limited — it operates within the existing AP framework rather than as a separate post-decode phase.

### Concurrency

WSJT-X 3.0 uses OpenMP to split the decode frequency band across up to 12 threads — each thread runs the full multi-pass decode pipeline on its slice of the spectrum independently. The number of threads is auto-calculated from CPU core count (or manually set via `--MTft8-threads`). Within each thread, candidate decoding and subtraction are sequential. FFTs use single-threaded FFTW despite initializing thread support.

wsjtr parallelizes differently: rayon is used for data parallelism within the pipeline — spectrogram computation, candidate sync evaluation, and diversity trials all run across cores. The decode loop itself (candidate → ft8b → LDPC → subtract) is sequential since subtraction is order-dependent, but the expensive FFT and correlation work leading up to it is parallelized. wsjtr does not split by frequency band.

### Diversity decoding

wsjtr supports running multiple independent decode trials with varied parameters (`--diversity N --diversity-strategy candidate-sort`), then taking the union of all results. Different trials may use different candidate sort orders or subtraction orderings (`--subtraction-order snr_asc`), which can recover signals that are masked by a particular subtraction sequence. This is unique to wsjtr — WSJT-X has no equivalent.

### What wsjtr does not implement (yet)

- **AP strategies 3–6**: WSJT-X supports six a priori decoding types, selected based on QSO progress (which Tx message is active). Types 1 (CQ) and 2 (MyCall) constrain 32 of 77 bits. Types 3–6 require knowing *both* callsigns and constrain 61–77 bits: type 3 fixes MyCall+DxCall (61 bits), types 4/5/6 fix the entire message to MyCall+DxCall+RRR/73/RR73 respectively. wsjtr only implements types 1 and 2, since types 3–6 require QSO state tracking from a GUI or sequencer.
- **Contest modes**: WSJT-X has specialized HOUND/FOX (ncontest=7) and contest-specific AP modes (Field Day, RTTY, etc.). wsjtr has no contest support.
- **Multi-cycle decoding**: WSJT-X 3.0 adds `nft8cycles` (1–3), where each cycle is a full 3-pass decode+subtract sequence — so up to 9 total passes. wsjtr's outer passes achieve a similar effect but without the cycle abstraction.
- **Other protocols**: WSJT-X decodes JT9, JT65, JT4, Q65, FST4, MSK144, and WSPR. wsjtr only handles FT4/FT8.
- **Waterfall / GUI integration**: WSJT-X computes and shares symbol spectra (`ss` array) for the waterfall display. wsjtr has no GUI (though `supplement_wsjtx.py` provides a supplemental decode display alongside WSJT-X).