snac-sweep 0.1.3

Sweep batch size, chunk size, and session count to benchmark ONNX decoder latency and RTF.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

# snac-sweep

Benchmark sweep for [SNAC](https://github.com/hubertsiuzdak/snac)'s ONNX decoder inference: varies sessions (parallel sessions), chunk size (sequence length without context), and batch size to find the largest throughput (sessions × batch) under real-time factor (RTF) < 1.

## Install

```bash
cargo install snac-sweep
```

## Usage

```bash
snac-sweep --context-size <N> --max-chunk-size <M> --model <onnx-file>
```

- context-size (default `3`): number of context windows added to sequence length.
- max-chunk-size (default `3`): max chunk size to sweep (seq_len from 1..=M).
- model (default `decoder_model_fp16.onnx`): model file under https://huggingface.co/onnx-community/snac_24khz-ONNX/tree/main/onnx.

RTF calculation is based on 24kHz sample rate (each chunk is 2048 samples).

## Example

```bash
snac-sweep --context-size 3 --max-chunk-size 5 --model decoder_model_fp16.onnx
```