1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
//! Paired comparison of opus1 with libopus 1.6.1.
//!
//! Each item is coded by both encoders with the same settings; libopus
//! decodes both streams; `opus_compare` (and MOC for speech) scores each
//! decoded signal against the clean reference; the tool reports the ratio
//! of the two errors per item, geometric means per condition with bootstrap
//! confidence intervals, the worst items, and the bytes each encoder spent.
//! Design: section 4.4 of the evaluation design note (A0).
//!
//! ```text
//! cargo run --release --example eval -- list [--set NAME]
//! cargo run --release --example eval -- run [--set NAME] [--items FILE] [--corpus DIR]
//! [--filter S] [--item-filter S] [--max-items N] [--json PATH] [--md PATH]
//! [--text PATH] [--baseline JSON] [--degrade KIND] [--write-wav DIR]
//! [--decoder libopus|opus1] [--threads N] [--strict]
//! cargo run --release --example eval -- selftest [--set NAME] [--threads N]
//! cargo run --release --example eval -- conformance [--vectors DIR] [--threads N]
//! cargo run --release --example eval -- moc REF.wav TEST.wav [--apply-vad]
//! cargo run --release --example eval -- corpus-index --root DIR --out FILE
//! [--per-group N] [--class CLASS] [--corpus DIR]
//! cargo run --release --example eval -- merge RUN.json TIER2.jsonl [--json PATH]
//! ```
//!
//! The sets are `quick` (default; no download needed), `selftest`, `tsp`,
//! `tsp_rates`, `tsp_stereo`, `music` and `baseline` (see `sets.rs`). The corpus root is `--corpus`,
//! else `OPUS1_CORPUS`, else `<repository>/corpus`.
//!
//! Gates only warn: `run` exits with 1 when a job fails (for example on a
//! final range mismatch), and with `--strict` also when a gate fails.