1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
/*
* SPDX-License-Identifier: MIT
* Copyright (c) 2023 - 2026. The DeepCausality Authors and Contributors. All Rights Reserved.
*/
//! Mean, variance and standard deviation over a sample.
//!
//! The variance here is the corrected (`n − 1`) form, and it is the only one built. Every variance
//! in this workspace applies Bessel's correction; there is no population `÷n` caller, and this
//! crate implements only what something calls.
//!
//! # An intermediate that fails the working scalar is a defect, not a limitation
//!
//! Every reduction below is written so that an answer the working scalar can hold is returned as a
//! number rather than as an artefact of how it was formed. An intermediate fails in two ways, and
//! each has its own answer here.
//!
//! It can leave the type. A rescaled second pass runs where the sum saturated: it costs nothing on
//! the ordinary path, and on the extreme one it is the difference between `mean([3e38, 3e38])`
//! returning `3e38` at `BFloat16` and returning `+∞`.
//!
//! It can also stall inside the type, which is quieter and bites far sooner. A running total grows
//! while its addends do not, and once the total passes the point where an addend falls below its
//! last place, every further addend rounds away — no infinity, no error, just a total that stops
//! moving. Every sum here is therefore formed as a balanced tree by `pairwise_sum`, including the
//! rescaled passes, which are just as exposed: a left-to-right `mean([3e38; 1000])` at `BFloat16`
//! returns 7.68e37 not because the sum overflows but because the *rescaled* sum of a thousand ones
//! stalls at 256.
use crateStatsError;
use cratepairwise_sum;
use RealField;
use FromPrimitive;
/// The arithmetic mean.
///
/// Refuses the empty sample: the mean of no observations is not zero, it is undefined, and
/// returning zero would put a plausible number where there is no answer.
///
/// # The sum's reach is not the mean's
///
/// The mean of a finite sample is a convex combination of it, so it is bounded in magnitude by the
/// largest observation and is always representable. The sum is not: two observations at `3e38`
/// overflow `BFloat16` while their mean does not. Where the sum saturates, the mean is
/// re-formed through the largest magnitude in the sample — every scaled observation then lies in
/// `[−1, 1]`, the scaled sum is bounded by `n`, and multiplying the scaled mean back by the scale
/// last cannot leave the type because the result is bounded by the scale itself.
///
/// A sample that carries a non-finite observation is a different case, and it keeps the saturated
/// sum: `±∞` and `NaN` are the answer there rather than an artefact of the intermediate, and the
/// rescaling would turn an honest infinity into a `NaN`.
/// A count on the real axis, or a typed error when the working scalar cannot hold it.
/// The largest `|xᵢ − centre|` in the sample, or zero for an empty one.
///
/// The scale a rescaled reduction divides by. Passing `centre = 0` gives the largest magnitude in
/// the sample itself.
pub
/// `Σ(xᵢ − centre)² / denominator`, formed so that a representable answer comes back as one.
///
/// The unscaled deviations first, because dividing before squaring costs precision where it is not
/// needed. A single `d²` can leave the type while
/// the quotient stays well inside it — one deviation of `1e20` among a hundred observations
/// overflows `BFloat16`'s `3.4e38` when squared, and `Σd²/(n − 1)` is then about `1e38`, which the
/// type holds — so where the sum saturates the deviations are divided by the largest of them
/// before squaring. The scaled sum lies in `[1, n]`, and the two multiplications back are ordered
/// so the quotient is applied first: `(r/denominator · s) · s` stays inside the type whenever the
/// answer does.
/// The sample variance, with Bessel's correction: `Σ(xᵢ − x̄)² / (n − 1)`.
///
/// Refuses a sample of fewer than two observations. With one, the divisor is zero and there is no
/// dispersion to estimate.
/// The sample standard deviation: the square root of [`variance`].
/// The population variance: `Σ(xᵢ − x̄)² / n`.
///
/// The uncorrected form, dividing by `n` rather than `n − 1`. It is the variance *of the sample
/// itself*, treated as the whole population, rather than an estimate of the variance of a
/// population the sample was drawn from.
///
/// # When this is the one you want
///
/// Standardising features to zero mean and unit scale: the divisor cancels out of the
/// standardisation, and the population form is the convention every library uses there. Also any
/// case where the data *is* the population — every measurement in a run, rather than a draw from
/// something larger.
///
/// Unlike [`variance`], a single observation is not refused: a population of one has zero spread,
/// which is a fact about it rather than a missing estimate. Only the empty sample is refused,
/// because there is nothing to take the variance of.
/// The population standard deviation: the square root of [`population_variance`].