1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
//! Startup tuning that keeps the process's virtual address space bounded on
//! high-core-count machines, independent of `nproc`.
//!
//! # Background (issue #137)
//!
//! On a 128-core box with a restrictive `RLIMIT_AS` (`ulimit -v`, e.g. set
//! system-wide via PAM `limits.conf` on a shared host), `create()` could
//! panic on an allocation as small as ~220 MiB — nowhere near the configured
//! `--memory-limit`. The recovery buffer itself was never the problem;
//! [`RecoveryEncoder::new_smart`](crate::encoder::RecoveryEncoder::new_smart)
//! only ever allocates `recovery_count × slice_size` bytes for the pass
//! currently running, and `ops::ingest_files` streams input in fixed 8 MiB
//! chunks rather than reading whole files. What actually consumed the
//! address space was the *thread fan-out* that ran before either of those:
//!
//! - `#[tokio::main]`'s default multi-thread runtime spawns one worker
//! thread per core (~128 here), even though this binary's async layer
//! only ever drives one file through [`crate::ops::ingest_files`] at a
//! time — a handful of workers is enough.
//! - glibc's malloc creates up to `8 × ncores` per-thread arenas by
//! default, each reserving tens of MiB of address space (nearly RSS-free,
//! but counted in full against `RLIMIT_AS`, and never returned). With
//! ~128 tokio workers plus a rayon pool also sized to ~128 cores
//! (`performance_core_count`, in `main.rs`) both contending on malloc,
//! that alone can exceed a single-digit-GiB ceiling before a single PAR2
//! slice is accounted for.
//!
//! `parpar` and `par2cmdline` don't hit this because neither fans out
//! anywhere near that many OS threads.
//!
//! The fix here only caps the *incidental* fan-out (tokio's own worker/
//! blocking pools, and glibc's arena count) — never the rayon pool, which
//! is the genuinely CPU-bound stage and stays sized to physical cores via
//! `performance_core_count` for throughput.
/// Cap glibc malloc's per-thread arena reservations.
///
/// No-op on musl (no `mallopt`, no per-core arenas) and on non-Unix
/// targets. Must run before any thread is spawned: arenas are created
/// lazily on first contended allocation from a new thread and are never
/// reclaimed afterwards, so this has to be the first thing `main` does —
/// before the tokio runtime is built and before the rayon pool.
///
/// `M_ARENA_MAX = 2` trades some allocator contention for address space.
/// That's the right trade here: `parmesan`'s hot path allocates per slice,
/// not per byte, so arena contention is far below the noise floor, while
/// uncapped arenas are what turned a 128-core machine's normal core count
/// into an address-space-exhausting `create()` in issue #137.
/// Worker-thread cap for the tokio runtime, independent of core count.
///
/// `ops::ingest_files` processes one file at a time — one `spawn_blocking`
/// reader task feeding an await loop that hands slices to
/// [`crate::worker::Par2Worker`] — so it never needs more than a couple of
/// runtime workers. `#[tokio::main]`'s default of one worker per core buys
/// nothing here and was the largest single contributor to the fan-out in
/// issue #137.
const MAX_WORKER_THREADS: usize = 4;
/// Cap on tokio's blocking-thread pool.
///
/// A ceiling on worst-case growth, not a saving in the common case — tokio
/// creates blocking threads lazily on demand. Bounds the tail instead of
/// leaving tokio's default of 512.
const MAX_BLOCKING_THREADS: usize = 16;
/// Reduced stack size for tokio's own thread pools.
///
/// Rust's default is 2 MiB per thread; the async orchestration in this
/// binary (channel plumbing, `block_in_place` calls) needs nowhere near
/// that. The RS/SIMD work runs on rayon's threads, not these.
const THREAD_STACK_SIZE: usize = 1024 * 1024;
/// Build the multi-threaded tokio runtime `parmesan` runs on.
///
/// Replaces `#[tokio::main]`, whose defaults scale worker-thread count with
/// `nproc` — see the module docs. Must stay multi-threaded:
/// `ops::ingest_files` uses `block_in_place`, which panics on a
/// current-thread runtime.