1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
/*
* SPDX-License-Identifier: MIT
* Copyright (c) 2023 - 2026. The DeepCausality Authors and Contributors. All Rights Reserved.
*/
//! The log-sum-exp reduction, in its slice and two-term forms.
use cratepairwise_sum;
use Real;
/// `log Σ exp(xᵢ)`, computed through the maximum so the exponentials cannot overflow.
///
/// Total, with no error path: every input has an answer.
/// - The empty slice sums to zero and `log 0` is `−∞`, which is the identity for this reduction.
/// - A `NaN` anywhere propagates, whatever else the slice holds.
/// - A non-finite maximum saturates the result, because the shift `x − max` is undefined there.
///
/// # Why `NaN` is tested before the maximum
///
/// A sum over a set does not depend on the order of the set, and neither does its log. The
/// running maximum is not enough on its own to keep that: every comparison against a `NaN` is
/// false, so a `NaN` is dropped when it arrives after a larger value and *kept* when it arrives
/// first. With an infinity in the slice the two readings differ — `[NaN, +∞]` saturating on a
/// `NaN` maximum and `[+∞, NaN]` on an infinite one — and the reduction would answer `NaN` for one
/// ordering and `+∞` for the other. Testing for a `NaN` outright removes the dependence.
/// `log(eᵃ + e^b)` for two terms, computed through the larger so neither exponential overflows.
///
/// The two-term case is separate because its caller has two scalars rather than a slice, and
/// routing it through the slice form would allocate.
///
/// `e^a + e^b = e^b + e^a`, so this is symmetric in its arguments, including at the non-finite
/// ones. The `NaN` test comes before the "pick the larger" step for the reason it does in
/// [`log_sum_exp`]: `a > b` is false when either is a `NaN`, so the larger of `(NaN, +∞)` is the
/// infinity and the larger of `(+∞, NaN)` is the `NaN`, and saturating on that would answer `+∞`
/// one way round and `NaN` the other.