1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
//! # comnum-sim — Monte Carlo simulation campaigns
//!
//! A campaign evaluates a **scenario** (one random trial of a link) on every
//! **condition** of a **grid** (an SNR, a launch power, a modulation order, …), as many
//! times as a **stopping rule** requires, in parallel on every core, and reports every
//! metric with a confidence interval. The organisation follows PyTorch Lightning, so
//! that the experiment code stays short and the engineering (parallelism, seeds,
//! stopping, logging, progress) is written once:
//!
//! | PyTorch Lightning | `comnum_sim` | role |
//! |---|---|---|
//! | `LightningModule` | [`scenario::Scenario`] | what one trial does: `setup` once per condition, `trial` many times |
//! | `self.log(...)` | [`scenario::Record`] | the metrics a trial reports: error counts or values |
//! | `Trainer` | [`runner::Runner`] | runs the trials, in batches, on a thread pool |
//! | `seed_everything` | [`runner::Runner::seed`] | trial `t` of condition `c` uses `stream(seed, name, c, t)` |
//! | `max_epochs`, `EarlyStopping` | [`stopping::Stop`] | when a condition has enough trials (errors, precision, time) |
//! | `Callback` | [`callbacks::Callback`] | hooks at the start and end of the run, of every condition and batch |
//! | `Logger` | [`loggers::Logger`] | CSV (one row per condition, written as it completes) and JSON reports |
//! | `fast_dev_run` | [`runner::Runner::fast_dev_run`] | one batch per condition, to check a scenario quickly |
//!
//! | Module | Content |
//! |---|---|
//! | [`scenario`] | the [`Scenario`](scenario::Scenario) trait, [`from_fn`](scenario::from_fn), the [`Record`](scenario::Record) of a trial |
//! | [`grid`] | the conditions of a campaign: axes, zips, fixed values, explicit lists |
//! | [`runner`] | the [`Runner`](runner::Runner): batches of trials on a thread pool, seeds, results |
//! | [`stopping`] | the stopping rules [`Stop`](stopping::Stop) and their combinations |
//! | [`callbacks`] | the hooks of a run: progress bar, skipping the rest of a sweep |
//! | [`loggers`] | the CSV and JSON reports |
//! | [`stats`] | error rates with Wilson intervals, means with normal intervals |
//! | [`json`] | a minimal JSON writer for the reports |
//!
//! # Reproducibility
//!
//! Every trial draws its random numbers from its own stream
//! `stream(seed, scenario name, condition index, trial index)` ([`comnum_core::rng::stream`]),
//! the trials of a batch run in parallel, their records are merged **in trial order**,
//! and the stopping rule is evaluated after every batch. The results are therefore
//! identical whatever the number of threads (only a wall-time rule,
//! [`stopping::Stop::max_time`], depends on the machine).
//!
//! # Statistics
//!
//! An error count is an [`stats::ErrorRate`] (`errors / total`, Wilson score interval);
//! any other value is a [`stats::Mean`] (Welford's running mean and variance, normal
//! interval of the mean). Both merge exactly across trials.
//!
//! # Example: BER of QPSK over AWGN against the theory
//!
//! ```
//! use comnum_sim::prelude::*;
//! use comnum_mapping::{constellation::{Constellation, ConstellationKind}, source as sources};
//! use comnum_channel::awgn as channels;
//! use comnum_metrics as metrics;
//! use comnum_sim::scenario::from_fn;
//!
//! // a scenario written as a closure: no per-condition state
//! let qpsk = from_fn("qpsk", |cond, rng, rec| {
//! let c = Constellation::qam(4)?.normalized();
//! let bits = sources::bits(2000, rng);
//! let rx = channels::awgn_ebn0(&c.map_bits(&bits)?, cond.f64("ebn0_db")?, 2.0, 1.0, rng);
//! rec.errors("ber", metrics::bit_errors(&c.demap_bits(&rx), &bits)? as u64, 2000);
//! Ok(())
//! });
//! let grid = Grid::new().axis("ebn0_db", [0.0, 2.0, 4.0]);
//! let res = Runner::new()
//! .seed(7)
//! .batch(16)
//! .stop(Stop::min_errors("ber", 500).or(Stop::max_trials(2000)))
//! .run(&qpsk, &grid)
//! .unwrap();
//! for r in &res {
//! let eb = r.condition.f64("ebn0_db").unwrap();
//! let ber = r.metric("ber").unwrap();
//! let (lo, hi) = ber.interval(0.99);
//! let theory = metrics::theory_ber(ConstellationKind::Qam, 4, eb).unwrap();
//! assert!(lo <= theory && theory <= hi, "{eb} dB: {theory} not in [{lo}, {hi}]");
//! }
//! ```
/// Version of the library, recorded in every run.
pub const VERSION: &str = env!;
/// Commit of the library at build time, recorded in every run: `<hash>-dirty` when the
/// tracked sources of the library (`crates/`, `bindings/`, `Cargo.toml`, `Cargo.lock`)
/// have uncommitted changes, staged or not, `unknown` outside a Git checkout. The build
/// script reruns when `HEAD`, the branch it points to, the index or one of these sources
/// changes, so that the value is never stale.
pub const COMMIT: &str = env!;
/// The items used by most campaigns.