Skip to main content

gam_terms/basis/
constant_curvature_smooth.rs

1//! Constant-curvature (`M_κ`) smooth term: basis + penalty over the
2//! κ-stereographic chart (#944, stage 3 step 1).
3//!
4//! The term is the κ-generic sibling of the intrinsic-S² Wahba smooth
5//! (`sphere_spec.rs` / `build_spherical_spline_basis`): a reproducing-kernel
6//! basis on a center set, with the kernel Gram on the centers as the RKHS
7//! roughness penalty and a coefficient-space sum-to-zero constraint for
8//! identifiability. Where the Wahba smooth hard-codes S² (lat/lon chart,
9//! Legendre kernels), this term takes the geometry from
10//! [`gam_geometry::constant_curvature::ConstantCurvature`] at an explicit
11//! curvature κ, so one construction covers the whole interpolation
12//! `S^d(1/√κ) → ℝ^d → H^d(1/√−κ)` through κ = 0.
13//!
14//! # Kernel
15//!
16//! `k_{κ,ℓ}(x, y) = ℓ·(exp(−d_κ(x, y)/ℓ) − 1)` — the geodesic-exponential
17//! kernel in its CONTRAST gauge, where `d_κ` is the exact constant-curvature
18//! geodesic distance in the κ-stereographic chart. The geodesic distance is a
19//! kernel of conditionally negative type on all three constant-curvature space
20//! forms (Schoenberg 1942 for `S^d`; classical CND of `‖·‖` on `ℝ^d`;
21//! Faraut–Harzallah 1974 for `H^d`), so `exp(−c·d_κ)` is positive definite for
22//! every `c > 0` and every κ, and the Gram on distinct centers is strictly PD
23//! on the sum-to-zero frame — which is exactly what the RKHS penalty
24//! construction needs. At κ = 0 the chart carries the doubled gauge
25//! (`metric 4δ`, `d_0(x, y) = 2‖x − y‖`), so the κ = 0 term is the Euclidean
26//! exponential (Matérn-½) kernel smooth with effective Euclidean range `ℓ/2`.
27//!
28//! The `ℓ` factor and the subtracted `1` are both invisible to the model —
29//! `z` annihilates constants and `λ` absorbs a positive scale — and both are
30//! load-bearing anyway, because the gauge is what decides whether the range
31//! coordinate is confounded with `ρ` and whether the criterion survives its own
32//! rounding at large `ℓ` (gam#2747). The derivation, with the measurements that
33//! forced it, is on [`constant_curvature_kernel_matrix`]. Its consequence for
34//! the family: `k → −d_κ` as `ℓ → ∞`, so the range's far face is the
35//! geodesic-distance kernel — an ordinary member of the family's own closure,
36//! not a degeneracy.
37//!
38//! # The exponent has TWO coordinates, and BOTH are estimated (gam#2747)
39//!
40//! `d_κ/ℓ` carries a curvature and a range, and they are strongly confounded:
41//! to first order `d_κ = d_0·(1 + κ·a(x,y))`, so the MEAN of `a` over the
42//! evaluated pairs acts exactly like a rescaling of `ℓ` and only the VARIATION
43//! of `a` is genuine curvature. A construction that fits κ while pinning ℓ to a
44//! heuristic therefore does not estimate curvature at all — it lets κ absorb
45//! whatever range correction the heuristic got wrong, which is monotone in one
46//! direction, so `V_p(κ)` rails.
47//!
48//! That was measured on truths planted inside the fitted span (gam#2747): with
49//! `ℓ` pinned at the auto `ℓ_ref`, the criterion recovers the planted κ⋆ only
50//! when the truth's own radial length scale happens to BE `ℓ_ref`. At half or
51//! twice that range it rails at a box endpoint, reports the wrong SIGN, or —
52//! on genuinely FLAT data — reports a confident interior `κ̂ = ∓0.94`. With
53//! `ℓ` profiled it lands within 0.19 of κ⋆ in all nine cells of a
54//! 3 curvatures × 3 ranges sweep (median 0.07), with no rails and no sign
55//! inversions, and `ℓ̂` recovers the planted range to 3%.
56//!
57//! So the smooth exposes the ψ-vector `(κ, η)` with `η = ln ℓ` — the same
58//! treatment the Matérn / Duchon / TPS siblings already give their range,
59//! extended by the signed curvature coordinate that keeps κ = 0 interior.
60//! Two prior attempts to remove the confounding by CONSTRAINT rather than by
61//! estimation are retired by this: `#1059`'s mean-geodesic-distance
62//! normalization and `#944`'s fill-invariant `L(κ)` both pin one scalar summary
63//! of the design, which selects a one-dimensional curve through the `(κ, ℓ)`
64//! plane a priori. On such a curve `dV/dκ = V_κ + V_ℓ·L′(κ)`, and the second
65//! term vanishes only if `ℓ_ref` was already optimal; on the profile curve it
66//! vanishes identically by the envelope theorem. `#1464`'s separate penalty
67//! length `L_S(κ)` is retired with them: it made `S` the Gram of a DIFFERENT
68//! kernel than the design's, so the penalty was no longer the RKHS roughness
69//! of the function it penalized.
70//!
71//! # ψ-differentiability contract (what the outer stage consumes)
72//!
73//! Every ψ-moving piece of this construction is differentiable in `(κ, η)` in
74//! closed form, and every ψ-FIXED piece is documented as such:
75//!
76//! - **Centers are ψ-fixed.** Center selection runs in chart coordinates
77//!   (farthest-point / k-means / user-provided) and deliberately consults
78//!   neither κ nor ℓ, so `∂(centers)/∂ψ ≡ 0` and the design moves with ψ only
79//!   through the kernel. A ψ-dependent center rule would add an uncontrolled,
80//!   non-smooth term to the design drift.
81//! - **The constraint transform `z` is ψ-fixed.** Uniform coefficient
82//!   weights; at fit time the global identifiability pipeline composes the
83//!   parametric orthogonalization onto it and the result is FROZEN
84//!   (mirroring `SphericalSplineIdentifiability::FrozenTransform`, #532), so
85//!   the predict/ψ-trial rebuild replays the same `z` verbatim.
86//! - **The kernel has exact ψ-jets.** With `q = d_κ(x,y)·e^{−η}` and the
87//!   Tower4-exact, FD-gated `distance_kappa_jet` supplying `(d, d′, d″)`:
88//!   `q_κ = d′/ℓ`, `q_κκ = d″/ℓ`, `q_η = −q`, `q_κη = −q_κ`, `q_ηη = q`, and
89//!   `K = e^{−q}` gives `K_a = −q_a K`, `K_ab = (q_a q_b − q_ab) K` — see
90//!   [`constant_curvature_kernel_psi_jets`]. The realized drifts follow by the
91//!   ψ-fixed transforms: `∂X/∂ψ_a = (∂K_dc/∂ψ_a)·z` and
92//!   `∂S/∂ψ_a = symm(zᵀ(∂K_cc/∂ψ_a)z)`, with no normalization quotient to
93//!   propagate (the RKHS penalty ships raw, `normalization_scale = 1`).
94//! - **Available but not yet consumed:** `log_map_kappa_jet` /
95//!   `exp_map_kappa_jet` cover future geodesic/normal-coordinate basis
96//!   variants (e.g. tangent-space designs); the distance jet is the only one
97//!   this kernel construction needs.
98
99use ndarray::{Array1, Array2, ArrayView2, Axis};
100use rayon::prelude::*;
101use serde::{Deserialize, Serialize};
102
103use gam_geometry::constant_curvature::{ConstantCurvature, distance_kappa_jet};
104
105use super::{
106    ActivePenalty, BasisBuildResult, BasisError, BasisMetadata, BasisPsiDerivativeBundle,
107    BasisPsiDerivativeResult, BasisPsiSecondDerivativeResult, CenterStrategy, CenterStrategyKind,
108    ConstructiveQuadratic, PenaltyCandidate, PenaltySource, center_strategy_kind,
109    filter_penalty_candidates, normalize_penalty, select_centers_by_strategy,
110    weighted_coefficient_sum_to_zero_transform,
111};
112
113/// Realized-design identifiability policy for the constant-curvature smooth.
114/// Mirrors [`super::SphericalSplineIdentifiability`] (#532): the fit-time
115/// center-space sum-to-zero `z` gets the parametric orthogonalization composed
116/// onto it by the global identifiability pipeline, and the composed transform
117/// is frozen here so predict-time (and future per-ψ-trial) rebuilds replay it
118/// verbatim instead of recomputing `z` from the centers.
119#[derive(Debug, Clone, Serialize, Deserialize, Default)]
120pub enum ConstantCurvatureIdentifiability {
121    /// Fit-time default: uniform-weight coefficient sum-to-zero over the
122    /// centers (`Σ_j α_j = 0`), then global parametric residualization.
123    #[default]
124    CenterSumToZero,
125    /// Predict-time replay: the frozen composed transform captured at fit
126    /// time. `transform.nrows()` equals the number of centers.
127    FrozenTransform { transform: Array2<f64> },
128}
129
130/// Constant-curvature smooth configuration (`curv(x, z, kappa = …)`).
131///
132/// The chart inputs are the raw feature columns interpreted as
133/// κ-stereographic chart coordinates: any finite point for κ ≥ 0, the open
134/// ball `‖x‖ < 1/√(−κ)` for κ < 0. The default κ = 0 reproduces a Euclidean
135/// exponential-kernel smooth (in the doubled κ = 0 chart gauge), so the term
136/// is safe to use as a drop-in flat smooth until κ becomes a fitted
137/// ψ-coordinate.
138#[derive(Debug, Clone, Serialize, Deserialize)]
139pub struct ConstantCurvatureBasisSpec {
140    /// Center/knot selection strategy in chart coordinates. Deliberately
141    /// κ-independent (see the module-level κ-contract).
142    pub center_strategy: CenterStrategy,
143    /// Sectional curvature κ of the latent/feature geometry. Fixed at build
144    /// time; when [`Self::kappa_fixed`] is `false` the later ψ-channel stage
145    /// promotes it to a fitted outer coordinate consuming this module's exact
146    /// κ-jets, and this field is only the seed. When `kappa_fixed` is `true`
147    /// this value is the user's PINNED sectional curvature and the outer loop
148    /// must hold it constant (never re-derive it).
149    pub kappa: f64,
150    /// Did the user explicitly pin the sectional curvature (`curv(.., kappa=K)`)?
151    ///
152    /// This is the mgcv-`sp=` convention applied to κ: an explicit `kappa=`
153    /// selects a FIXED geometry (`Sᵈ` for κ>0, `ℝᵈ` for κ=0, `Hᵈ` for κ<0) and
154    /// the fit builds/keeps the design and penalty at exactly that κ; an OMITTED
155    /// `kappa=` leaves κ free, seeded at [`Self::kappa`] (default 0), for the
156    /// #944/#1464 outer ψ-coordinate estimation to fit. The two paths must never
157    /// be confused: honoring the pin is the whole contract of a fixed-curvature
158    /// smooth (gam#2152), while the estimation path is the whole point of the
159    /// κ-inference subsystem. Defaults to `false` (estimate) so the estimand
160    /// machinery and every serialized pre-#2152 model keep their behaviour.
161    #[serde(default)]
162    pub kappa_fixed: bool,
163    /// Geodesic kernel range ℓ in `K_κ = exp(−d_κ/ℓ)`. The `0.0` sentinel
164    /// requests the κ-independent auto initialization
165    /// ([`realized_constant_curvature_length_scale`]); the realized value is
166    /// persisted in [`BasisMetadata::ConstantCurvature`] and frozen back into
167    /// the spec for predict-time replay.
168    ///
169    /// When [`Self::length_scale_fixed`] is `false` this is only the SEED: the
170    /// range is the smooth's second outer ψ-coordinate (`η = ln ℓ`) and is
171    /// estimated jointly with κ, because a pinned range makes κ absorb the
172    /// range error instead of measuring curvature (gam#2747, module docs).
173    pub length_scale: f64,
174    /// Did the user explicitly pin the kernel range (`curv(.., length_scale=L)`)?
175    ///
176    /// The mgcv-`sp=` convention that [`Self::kappa_fixed`] already applies to
177    /// the curvature, applied to the range: an explicit `length_scale=` selects
178    /// a FIXED kernel resolution and the fit honors it verbatim; an OMITTED
179    /// `length_scale=` leaves `η = ln ℓ` free for the gam#2747 outer estimation,
180    /// seeded at the auto rule. Defaults to `false` (estimate), so a spec that
181    /// merely had its realized ℓ frozen back into it after a fit is not
182    /// mistaken for a user pin on the next fit.
183    #[serde(default)]
184    pub length_scale_fixed: bool,
185    /// Add the ridge-like shrinkage penalty alongside the RKHS Gram penalty.
186    pub double_penalty: bool,
187    /// Realized-design identifiability policy (see type docs).
188    #[serde(default)]
189    pub identifiability: ConstantCurvatureIdentifiability,
190}
191
192impl Default for ConstantCurvatureBasisSpec {
193    fn default() -> Self {
194        Self {
195            center_strategy: CenterStrategy::FarthestPoint { num_centers: 50 },
196            kappa: 0.0,
197            kappa_fixed: false,
198            length_scale: 0.0,
199            length_scale_fixed: false,
200            // No double-penalty ridge by default (#1464). The RKHS Gram penalty
201            // zᵀKz is strictly PD/full-rank on distinct centers, so it already
202            // regularizes every coefficient direction — the ridge `I` adds no
203            // stability. Worse, `I` is curvature-BLIND: with its own λ it absorbs
204            // the data fit independently of κ. Curvature-sign identification
205            // remains the separate #1464 problem; an extra curvature-blind
206            // ridge cannot resolve it.
207            double_penalty: false,
208            identifiability: ConstantCurvatureIdentifiability::CenterSumToZero,
209        }
210    }
211}
212
213/// Validate that every row of `points` is finite and inside the
214/// κ-stereographic chart: `1 + κ‖x‖² > 0`, the open ball `‖x‖ < 1/√(−κ)` for
215/// κ < 0 and vacuous for κ ≥ 0.
216///
217/// **The vacuity on the κ ≥ 0 branch is not a gap, and this doc used to read as
218/// if it were** (gam#2687 quoted the old wording — *"automatic for κ ≥ 0"* — as
219/// evidence that the code did not implement its own comment). The two branches
220/// have different constraints, of different ARITY, enforced at different sites:
221///
222/// * **κ < 0 — a PER-POINT gauge, enforced here.** `λ(p) = 1 + κ‖p‖²` is the
223///   conformal factor's denominator; it vanishes when `p` reaches the Poincaré
224///   ball's boundary. One point is enough to violate it, so a per-row scan is
225///   the right check and this function is where it belongs. It runs on data
226///   **and** centers, because both are points the kernel evaluates.
227/// * **κ > 0 — a PER-PAIR fold, enforced by the κ box.** `λ` really cannot
228///   vanish for κ ≥ 0, but the quantity that does is
229///   `D = 1 + 2κ⟨x,c⟩ + κ²‖x‖²‖c‖²`, the Möbius denominator of
230///   `w = (−x) ⊕_κ c`, which is `(1 − κ‖x‖‖c‖)²` for an anti-aligned pair and
231///   vanishes at the antipodal fold `κ = 1/(‖x‖‖c‖)`. No per-point predicate can
232///   see it — it is a property of a PAIR — so the retreat is taken upstream, by
233///   [`constant_curvature_kappa_bounds`](crate::smooth::constant_curvature_kappa_bounds),
234///   which caps `|κ|` at `F/R²` over `R = max‖p‖` on `data ∪ centers` (gam#2716:
235///   over `data` alone, a user-provided center past `2·max‖x‖` put the box past
236///   the fold and made it doubly covered).
237///
238/// So there is exactly one wall per branch and each is checked against its own
239/// geometry; neither is the other mirrored. Gated by
240/// `spherical_branch_folds_at_kappa_r2_one_so_the_kappa_window_is_symmetric_2687`
241/// (gam-geometry) and by `crate::smooth::constant_curvature_kappa_box_tests`,
242/// which drives both endpoints of the shipped box through this function and
243/// through the shipped `distance`.
244pub(crate) fn validate_chart_points(
245    points: ArrayView2<'_, f64>,
246    kappa: f64,
247    what: &str,
248) -> Result<(), BasisError> {
249    for (i, row) in points.outer_iter().enumerate() {
250        let mut nx2 = 0.0_f64;
251        for &v in row.iter() {
252            if !v.is_finite() {
253                crate::bail_invalid_basis!(
254                    "constant-curvature {what} row {i} has a non-finite coordinate"
255                );
256            }
257            nx2 += v * v;
258        }
259        if 1.0 + kappa * nx2 <= 0.0 {
260            crate::bail_invalid_basis!(
261                "constant-curvature {what} row {i} lies outside the κ-stereographic chart \
262                 (need 1 + κ·‖x‖² > 0; got κ = {kappa}, ‖x‖² = {nx2}); for κ < 0 the chart is \
263                 the open ball ‖x‖ < 1/√(−κ)"
264            );
265        }
266    }
267    Ok(())
268}
269
270/// `φ(u) = e^{−u}(1 + u) − 1` — the `∂/∂η` shape factor of the normalized
271/// kernel, evaluated without cancellation.
272///
273/// `φ` vanishes to SECOND order at `u = 0` (`φ = −u²/2 + u³/3 − …`) while both
274/// of its terms are `O(1)`, so the direct form loses `2·log₁₀(1/u)` digits as
275/// `u → 0` — which is exactly the regime the range coordinate walks into. The
276/// series is `−Σ_{m≥2} (−1)^m (m−1) u^m / m!`, used below `u = 1/2` where its
277/// terms are monotone and the truncation at `m = 20` is `< 2⁻²⁰/20! ≈ 5e-25`.
278#[inline]
279fn eta_shape(u: f64) -> f64 {
280    if u >= 0.5 {
281        return (-u).exp() * (1.0 + u) - 1.0;
282    }
283    let mut term = u * u; // u^m / m! at m = 2, times 2!
284    let mut factorial = 2.0_f64;
285    let mut sum = 0.0_f64;
286    let mut sign = 1.0_f64; // (−1)^m at m = 2
287    for m in 2..=20u32 {
288        if m > 2 {
289            term *= u;
290            factorial *= f64::from(m);
291            sign = -sign;
292        }
293        sum -= sign * f64::from(m - 1) * term / factorial;
294    }
295    sum
296}
297
298/// `χ(u) = e^{−u}(1 + u + u²) − 1` — the `∂²/∂η²` shape factor.
299///
300/// Same second-order zero at the origin (`χ = u²/2 − 2u³/3 + …`) and the same
301/// series treatment, with coefficients `(m−1)²/m!`. Away from the origin the
302/// direct form is used; `χ` has a genuine root near `u = 2.15`, where relative
303/// precision is unavailable to any formula and irrelevant — the block is summed
304/// over pairs, so absolute accuracy is what the Hessian consumes.
305#[inline]
306fn eta2_shape(u: f64) -> f64 {
307    if u >= 0.5 {
308        return (-u).exp() * (1.0 + u + u * u) - 1.0;
309    }
310    let mut term = u * u;
311    let mut factorial = 2.0_f64;
312    let mut sum = 0.0_f64;
313    let mut sign = 1.0_f64;
314    for m in 2..=20u32 {
315        if m > 2 {
316            term *= u;
317            factorial *= f64::from(m);
318            sign = -sign;
319        }
320        let weight = f64::from(m - 1) * f64::from(m - 1);
321        sum += sign * weight * term / factorial;
322    }
323    sum
324}
325
326/// The model kernel at one pair: `k = ℓ·(e^{−d/ℓ} − 1)`, evaluated as
327/// `ℓ·expm1(−d/ℓ)`.
328#[inline]
329pub(crate) fn constant_curvature_kernel_scalar(distance: f64, length_scale: f64) -> f64 {
330    length_scale * (-distance / length_scale).exp_m1()
331}
332
333/// `k_κ(data, centers)` — the realized constant-curvature model kernel matrix
334/// `ℓ·(e^{−d_κ(x_i,c_j)/ℓ} − 1)`, evaluated as `ℓ·expm1(−d_κ/ℓ)`.
335///
336/// # Why this is not `exp(−d_κ/ℓ)` (gam#2747)
337///
338/// The kernel is only ever consumed through the coefficient sum-to-zero frame
339/// `z`: the realized design is `K z` and the realized penalty is `zᵀ K z`. `z`
340/// annihilates constants, so the construction is invariant under `K → K + c1ᵀ`
341/// for any per-row `c`, and multiplying `K` by a positive scalar is absorbed
342/// exactly by the smoothing parameter (the prior on the fitted function is
343/// `(1/λ)·X S⁻ Xᵀ`, invariant under `(X, S, λ) → (aX, aS, aλ)`). So `exp(−d/ℓ)`
344/// and `ℓ·(e^{−d/ℓ} − 1)` are the SAME model in two gauges — and the gauge is
345/// not free, because two things are decided by it.
346///
347/// **It decides whether the range coordinate is confounded with the smoothing
348/// parameter.** `exp(−d/ℓ)z = −(1/ℓ)Dz + O(1/ℓ²)`, so the realized design and
349/// penalty both collapse like `1/ℓ` and `λ̂` has to chase them: measured on the
350/// κ = 1 sphere fixture, `ρ̂` falls one-for-one with `ln ℓ` over eleven decades
351/// (`−5.49 → −18.91` from `ℓ = 1` to `10⁶`) while the criterion value is
352/// unchanged to eight significant figures. A range search on an absolute `ρ`
353/// box therefore walks `ρ̂` into `RHO_LOWER` for no statistical reason. In this
354/// gauge `ρ̂` is FLAT (`−5.0978 ± 1e-4` over the same eleven decades).
355///
356/// **And it decides whether the criterion is a function of the data at all.**
357/// All of the model's range information lives in `K − 1`; forming it by
358/// subtracting `exp(−d/ℓ)` from an implicit `1` costs `log₁₀(ℓ/d)` significant
359/// digits, and the Gram then squares what is left. Measured on the same
360/// fixture, the `exp` gauge's REML value departs from the truth by **78.8 nats
361/// at the derived box top** `ℓ_hi = d_min/√ε = 2.53e6` and by 476 nats at
362/// `ℓ = 10⁸`, descending ~100 nats per decade into its own rounding with `edf`
363/// railed at `p` — which is what a range search reads when it "converges to an
364/// asymptote" at the box end. `expm1` forms `K − 1` directly and the departure
365/// is zero to eight figures at `ℓ = 10⁹`.
366///
367/// # Consequences of the gauge, all of them intended
368///
369/// * `k ≤ 0`, with `k = 0` exactly on coincident points. The realized penalty
370///   `zᵀkz = ℓ·zᵀe^{−d/ℓ}z` is still strictly positive definite on the frame —
371///   `−k` is a conditionally negative definite kernel, which is what makes it
372///   so — but the RAW `m × m` matrix is no longer PSD, so the builder forms the
373///   penalty from the RESTRICTED Gram rather than restricting a raw PSD one.
374/// * `k → −d_κ` as `ℓ → ∞`, exactly and with no cancellation. The `ℓ = ∞` face
375///   is the geodesic-distance kernel, an ordinary non-degenerate member of the
376///   family's own closure rather than a degenerate limit — which is what lets
377///   the range coordinate be compactified instead of walled.
378pub fn constant_curvature_kernel_matrix(
379    data: ArrayView2<'_, f64>,
380    centers: ArrayView2<'_, f64>,
381    kappa: f64,
382    length_scale: f64,
383) -> Result<Array2<f64>, BasisError> {
384    if data.ncols() != centers.ncols() {
385        crate::bail_dim_basis!(
386            "constant-curvature kernel dimension mismatch: data d={} centers d={}",
387            data.ncols(),
388            centers.ncols()
389        );
390    }
391    if !(length_scale.is_finite() && length_scale > 0.0) {
392        crate::bail_invalid_basis!(
393            "constant-curvature kernel needs a positive finite length_scale; got {length_scale}"
394        );
395    }
396    validate_chart_points(data, kappa, "data")?;
397    validate_chart_points(centers, kappa, "centers")?;
398    let manifold = ConstantCurvature::new(data.ncols(), kappa);
399    let mut out = Array2::<f64>::zeros((data.nrows(), centers.nrows()));
400    out.axis_iter_mut(Axis(0))
401        .into_par_iter()
402        .enumerate()
403        .try_for_each(|(i, mut row)| -> Result<(), BasisError> {
404            for (j, c) in centers.outer_iter().enumerate() {
405                let d = manifold.distance(data.row(i), c).map_err(|e| {
406                    BasisError::InvalidInput(format!(
407                        "constant-curvature distance failed at (row {i}, center {j}): {e}"
408                    ))
409                })?;
410                row[j] = constant_curvature_kernel_scalar(d, length_scale);
411            }
412            Ok(())
413        })?;
414    Ok(out)
415}
416
417/// The `(k, ∂k/∂ψ_a, ∂²k/∂ψ_a∂ψ_b)` tower of the raw (pre-constraint) model
418/// kernel matrix in BOTH outer coordinates, `ψ = (κ, η)` with `η = ln ℓ`.
419///
420/// Exact. `distance_kappa_jet` (Tower4, FD-gated in
421/// `geometry::constant_curvature`) supplies `(d, d′, d″)`; everything else is
422/// the chain rule on `k = ℓ·(e^{−q} − 1)` with `q = d_κ(x,y)·e^{−η}` (see
423/// [`constant_curvature_kernel_matrix`] for why the kernel carries the `ℓ`
424/// factor and the annihilated `−1`):
425///
426/// ```text
427///   k_κ  = −d′·e^{−q}                  k_κκ = e^{−q}·(d′²/ℓ − d″)
428///   k_η  =  ℓ·φ(q),  φ(u) = e^{−u}(1+u) − 1
429///   k_κη = −d′·q·e^{−q}
430///   k_ηη =  ℓ·χ(q),  χ(u) = e^{−u}(1+u+u²) − 1
431/// ```
432///
433/// Every block has a finite `ℓ → ∞` limit, and the two that vanish there do so
434/// through `φ` and `χ`, both of which have a second-order zero at the origin —
435/// so they are evaluated by series below `u = 1/2` rather than by the
436/// two-term difference, which would lose `2·log₁₀(ℓ/d)` digits exactly where
437/// the range coordinate is least identified. `k_κ`, `k_κκ` and `k_κη` are
438/// cancellation-free as written.
439///
440/// The `η` channel is what makes the range an ESTIMAND rather than a heuristic
441/// (gam#2747): with `ℓ` pinned, κ absorbs the range error and the profiled
442/// criterion rails. The realized design/penalty drifts follow by the ψ-fixed
443/// transforms `∂X/∂ψ_a = (∂K_dc/∂ψ_a)·z`,
444/// `∂S/∂ψ_a = symm(zᵀ(∂K_cc/∂ψ_a)z)`; the RKHS penalty ships raw
445/// (`normalization_scale = 1`), so no normalization quotient rule participates.
446#[derive(Clone, Debug)]
447pub struct ConstantCurvatureKernelPsiJets {
448    /// `k`.
449    pub value: Array2<f64>,
450    /// `∂k/∂κ`.
451    pub d_kappa: Array2<f64>,
452    /// `∂k/∂η`, `η = ln ℓ`.
453    pub d_eta: Array2<f64>,
454    /// `∂²k/∂κ²`.
455    pub d_kappa2: Array2<f64>,
456    /// `∂²k/∂κ∂η`.
457    pub d_kappa_eta: Array2<f64>,
458    /// `∂²k/∂η²`.
459    pub d_eta2: Array2<f64>,
460}
461
462/// Build [`ConstantCurvatureKernelPsiJets`] for one `(data, centers)` block.
463pub fn constant_curvature_kernel_psi_jets(
464    data: ArrayView2<'_, f64>,
465    centers: ArrayView2<'_, f64>,
466    kappa: f64,
467    length_scale: f64,
468) -> Result<ConstantCurvatureKernelPsiJets, BasisError> {
469    if data.ncols() != centers.ncols() {
470        crate::bail_dim_basis!(
471            "constant-curvature kernel-jet dimension mismatch: data d={} centers d={}",
472            data.ncols(),
473            centers.ncols()
474        );
475    }
476    if !(length_scale.is_finite() && length_scale > 0.0) {
477        crate::bail_invalid_basis!(
478            "constant-curvature kernel jets need a positive finite length_scale; got {length_scale}"
479        );
480    }
481    validate_chart_points(data, kappa, "data")?;
482    validate_chart_points(centers, kappa, "centers")?;
483    let manifold = ConstantCurvature::new(data.ncols(), kappa);
484    let n = data.nrows();
485    let m = centers.nrows();
486    let mut jets = ConstantCurvatureKernelPsiJets {
487        value: Array2::<f64>::zeros((n, m)),
488        d_kappa: Array2::<f64>::zeros((n, m)),
489        d_eta: Array2::<f64>::zeros((n, m)),
490        d_kappa2: Array2::<f64>::zeros((n, m)),
491        d_kappa_eta: Array2::<f64>::zeros((n, m)),
492        d_eta2: Array2::<f64>::zeros((n, m)),
493    };
494    let rows: Vec<(usize, Vec<[f64; 6]>)> = (0..n)
495        .into_par_iter()
496        .map(|i| -> Result<(usize, Vec<[f64; 6]>), BasisError> {
497            let mut row = Vec::with_capacity(m);
498            for (j, c) in centers.outer_iter().enumerate() {
499                let (d, d1, d2) = distance_kappa_jet(&manifold, data.row(i), c).map_err(|e| {
500                    BasisError::InvalidInput(format!(
501                        "constant-curvature distance κ-jet failed at (row {i}, center {j}): {e}"
502                    ))
503                })?;
504                let q = d / length_scale;
505                let decay = (-q).exp();
506                row.push([
507                    constant_curvature_kernel_scalar(d, length_scale),
508                    -d1 * decay,
509                    length_scale * eta_shape(q),
510                    decay * (d1 * d1 / length_scale - d2),
511                    -d1 * q * decay,
512                    length_scale * eta2_shape(q),
513                ]);
514            }
515            Ok((i, row))
516        })
517        .collect::<Result<Vec<_>, BasisError>>()?;
518    for (i, row) in rows {
519        for (j, entry) in row.into_iter().enumerate() {
520            jets.value[(i, j)] = entry[0];
521            jets.d_kappa[(i, j)] = entry[1];
522            jets.d_eta[(i, j)] = entry[2];
523            jets.d_kappa2[(i, j)] = entry[3];
524            jets.d_kappa_eta[(i, j)] = entry[4];
525            jets.d_eta2[(i, j)] = entry[5];
526        }
527    }
528    Ok(jets)
529}
530
531/// `(k, ∂k/∂κ, ∂²k/∂κ²)` — the κ slice of [`constant_curvature_kernel_psi_jets`],
532/// kept as its own entry point for callers that hold `ℓ` fixed.
533pub fn constant_curvature_kernel_kappa_jets(
534    data: ArrayView2<'_, f64>,
535    centers: ArrayView2<'_, f64>,
536    kappa: f64,
537    length_scale: f64,
538) -> Result<(Array2<f64>, Array2<f64>, Array2<f64>), BasisError> {
539    let jets = constant_curvature_kernel_psi_jets(data, centers, kappa, length_scale)?;
540    Ok((jets.value, jets.d_kappa, jets.d_kappa2))
541}
542
543/// Resolve the realized kernel range ℓ. An explicit positive `spec_length_scale`
544/// is used verbatim; the `0.0` sentinel auto-initializes from the median
545/// pairwise CHART distance among the centers, doubled to match the κ = 0
546/// chart gauge (`d_0 = 2‖Δ‖`).
547///
548/// This is a SEED unless the user pinned it (`length_scale_fixed`): the range is
549/// the smooth's second outer coordinate and the fit estimates it (gam#2747).
550/// The auto rule reads chart coordinates only — it never consults κ — so the
551/// seed and the derived search window
552/// ([`constant_curvature_length_scale_bounds`]) are both κ-FIXED, and the outer
553/// box does not move while the optimizer walks κ.
554pub fn realized_constant_curvature_length_scale(
555    centers: ArrayView2<'_, f64>,
556    spec_length_scale: f64,
557) -> Result<f64, BasisError> {
558    if spec_length_scale.is_finite() && spec_length_scale > 0.0 {
559        return Ok(spec_length_scale);
560    }
561    if spec_length_scale != 0.0 {
562        crate::bail_invalid_basis!(
563            "constant-curvature length_scale must be positive (or 0.0 for auto); got {spec_length_scale}"
564        );
565    }
566    let dists = center_chart_gauge_distances(centers)?;
567    let median = dists[dists.len() / 2];
568    if !(median.is_finite() && median > 0.0) {
569        crate::bail_invalid_basis!(
570            "constant-curvature auto length_scale failed: centers are degenerate \
571             (median pairwise chart distance = {median})"
572        );
573    }
574    Ok(median)
575}
576
577/// The sorted multiset of pairwise center distances in the κ = 0 doubled chart
578/// gauge (`d_0 = 2‖Δ‖`) — the single source both the auto `ℓ_ref` (its median)
579/// and the derived range window (its ends) are read from.
580fn center_chart_gauge_distances(centers: ArrayView2<'_, f64>) -> Result<Vec<f64>, BasisError> {
581    let m = centers.nrows();
582    if m < 2 {
583        return Err(BasisError::InsufficientColumnsForConstraint { found: m });
584    }
585    let mut dists: Vec<f64> = Vec::with_capacity(m * (m - 1) / 2);
586    for i in 0..m {
587        for j in (i + 1)..m {
588            let mut s = 0.0_f64;
589            for k in 0..centers.ncols() {
590                let dlt = centers[(i, k)] - centers[(j, k)];
591                s += dlt * dlt;
592            }
593            dists.push(2.0 * s.sqrt());
594        }
595    }
596    dists.sort_by(|a, b| a.partial_cmp(b).expect("finite chart distances"));
597    Ok(dists)
598}
599
600/// The scales the kernel actually evaluates: `(d_min⁺, d_max)` over the
601/// data→center **and** center→center pairs, in the κ = 0 doubled chart gauge,
602/// excluding the exact zeros that self-pairs contribute.
603///
604/// This is the same pair set the chart guard validates and the κ box takes its
605/// radius over (`data ∪ centers`, gam#2716) — one set, so a configuration that
606/// moves one of the smooth's two outer boxes moves the other consistently.
607pub fn constant_curvature_evaluated_scale_span(
608    data: ArrayView2<'_, f64>,
609    centers: ArrayView2<'_, f64>,
610) -> Result<(f64, f64), BasisError> {
611    if data.ncols() != centers.ncols() {
612        crate::bail_dim_basis!(
613            "constant-curvature scale span dimension mismatch: data d={} centers d={}",
614            data.ncols(),
615            centers.ncols()
616        );
617    }
618    let mut lo = f64::INFINITY;
619    let mut hi = 0.0_f64;
620    let mut observe = |a: ndarray::ArrayView1<'_, f64>, b: ndarray::ArrayView1<'_, f64>| {
621        let mut sum = 0.0_f64;
622        for k in 0..a.len() {
623            let delta = a[k] - b[k];
624            sum += delta * delta;
625        }
626        let d = 2.0 * sum.sqrt();
627        if d.is_finite() && d > 0.0 {
628            lo = lo.min(d);
629            hi = hi.max(d);
630        }
631    };
632    for x in data.outer_iter() {
633        for c in centers.outer_iter() {
634            observe(x, c);
635        }
636    }
637    for i in 0..centers.nrows() {
638        for j in (i + 1)..centers.nrows() {
639            observe(centers.row(i), centers.row(j));
640        }
641    }
642    if !(lo.is_finite() && lo > 0.0 && hi.is_finite() && hi >= lo) {
643        crate::bail_invalid_basis!(
644            "constant-curvature range window is undefined: the evaluated pairs carry no \
645             positive chart distance (d_min = {lo}, d_max = {hi})"
646        );
647    }
648    Ok((lo, hi))
649}
650
651/// DERIVED box `[ℓ_lo, ℓ_hi]` for the kernel range — the interval on which the
652/// realized design's GRAM is still resolvable in double precision, which is the
653/// only thing a box on this coordinate is entitled to enforce.
654///
655/// Two derivations were tried and measured before this one.
656///
657/// The first bounded `ℓ` by the scales the geometry contains, `[d_min⁺, d_max]`.
658/// That is wrong because the criterion `V(κ⋆, ℓ)` is sharply unimodal with an
659/// interior minimum that recovers the planted range and rises monotonically on
660/// both sides across four log-units — it walls the range in by itself — while
661/// its minimum sits OUTSIDE the center set's own span in a third of the planted
662/// cells. A window at the geometry's scales rails a coordinate the criterion
663/// handles perfectly well, trading one artificial constraint for another, which
664/// is exactly the mistake `#944` and `#1464` made in the κ direction.
665///
666/// The second put the wall at REPRESENTABILITY: `ℓ_lo = d_max/ln(1/MIN_POSITIVE)`
667/// and `ℓ_hi = d_min⁺/EPSILON`, the points at which `exp(−d_max/ℓ)` underflows
668/// and `1 − exp(−d_min⁺/ℓ)` rounds away. That is 39× too permissive at the
669/// bottom and seven orders too permissive at the top, because **the criterion
670/// does not evaluate the kernel — it evaluates a Cholesky of `H = XᵀX + λS`**,
671/// and the Gram SQUARES the design's dynamic range. At `ℓ_lo` so defined the
672/// design entries span 300 orders of magnitude, `H` is numerically singular,
673/// and the profile's derivatives come back at `10⁶`–`10⁸` — which is how a
674/// bounded outer solve ends up reporting `|Pg| = 8.2e6` against a stationarity
675/// bound of `1.1e-3` and failing its line search.
676///
677/// So the wall belongs where the linear algebra lives:
678///
679/// * **`ℓ_lo = d_max / (½·ln(1/ε))`** — with `q = d/ℓ`, the design's entries
680///   span `e^{−q_max}` and `XᵀX` spans its square, so a Cholesky resolves the
681///   Gram only while `e^{−2·q_max} ≥ ε`, i.e. `q_max ≤ ½·ln(1/ε) ≈ 18`.
682/// * **`ℓ_hi = d_max / (2√ε)`** — the far end is not a numerical wall at all,
683///   and the third derivation is what made that visible. `ℓ_hi` used to be
684///   `d_min⁺/√ε`, the point at which `1 − e^{−q_min}` rounds away and the Gram
685///   squares what is left. That loss was REAL, but it was a property of the
686///   `exp(−d/ℓ)` gauge rather than of the model: the design is `Kz` and `z`
687///   annihilates constants, so all the range information lived in `K − 1`, and
688///   forming it by subtraction cost `log₁₀(ℓ/d)` digits. The criterion built on
689///   it was measurably FALSE well before that wall — 78.8 nats of fabricated
690///   descent AT `ℓ_hi`, ~100 nats per decade past it, `edf` railed at `p`
691///   (gam#2747). The contrast gauge
692///   ([`constant_curvature_kernel_matrix`]) forms `K − 1` directly and the loss
693///   is gone, so nothing numerical bounds the range from above any more.
694///
695///   What bounds it is the MODEL: `k → −d_κ` as `ℓ → ∞`, so the far face of the
696///   range is the geodesic-distance kernel, and the departure from it is
697///   `d/(2ℓ)` relative, first order. Truncating the chart where the LARGEST
698///   evaluated pair is within `√ε` of its limit puts every entry of the design
699///   within the square root of machine precision of the limit design — past
700///   that point the range is not identified because there is nothing left to
701///   identify, and an estimate reported there is a statement about the model
702///   (`the kernel IS the geodesic distance`) rather than about the box. That is
703///   what [`RangeSolveOutcome`](../../../gam_models/index.html) declares rather
704///   than leaves to be inferred.
705///
706/// Both ends are read in the κ = 0 doubled gauge, the same gauge the auto
707/// `ℓ_ref` rule uses, so the box is κ-FIXED and does not move while the
708/// optimizer walks κ. It is still wide — some eight orders — and deliberately
709/// so: the criterion supplies the shape, this supplies only the wall.
710/// Bracketing the inner search is a separate concern and uses
711/// [`constant_curvature_evaluated_scale_span`] directly.
712pub fn constant_curvature_length_scale_bounds(
713    data: ArrayView2<'_, f64>,
714    centers: ArrayView2<'_, f64>,
715) -> Result<(f64, f64), BasisError> {
716    let (d_min, d_max) = constant_curvature_evaluated_scale_span(data, centers)?;
717    let gram_resolvable_efolds = 0.5 * -f64::EPSILON.ln();
718    let lo = d_max / gram_resolvable_efolds;
719    // The top end no longer reads `d_min` — the closest pair's contrast was the
720    // RETIRED gauge's failure mode, not this one's — but the span is still read
721    // as a pair because `d_min` guards the span's own validity and names the
722    // geometry in the refusal below.
723    let hi = d_max / (2.0 * f64::EPSILON.sqrt());
724    if !(lo.is_finite() && lo > 0.0 && hi.is_finite() && hi > lo) {
725        crate::bail_invalid_basis!(
726            "constant-curvature range box collapsed: [{lo}, {hi}] from an evaluated span of \
727             [{d_min}, {d_max}]"
728        );
729    }
730    Ok((lo, hi))
731}
732
733/// Build the constant-curvature reproducing-kernel smooth: realized design
734/// `K_κ(data, centers)·z`, RKHS penalty `zᵀ K_κ(centers, centers) z`, and the
735/// replayable [`BasisMetadata::ConstantCurvature`]. Structure mirrors the
736/// Wahba S² builder (`build_spherical_spline_basis`); geometry comes from
737/// `ConstantCurvature` at the spec's fixed κ.
738pub fn build_constant_curvature_basis(
739    data: ArrayView2<'_, f64>,
740    spec: &ConstantCurvatureBasisSpec,
741) -> Result<BasisBuildResult, BasisError> {
742    if data.ncols() == 0 {
743        crate::bail_invalid_basis!("constant-curvature smooth needs at least one feature column");
744    }
745    if !spec.kappa.is_finite() {
746        crate::bail_invalid_basis!("constant-curvature smooth needs a finite kappa");
747    }
748    validate_chart_points(data, spec.kappa, "data")?;
749    let centers = select_constant_curvature_centers(data, &spec.center_strategy)?;
750    if centers.nrows() < 2 {
751        return Err(BasisError::InsufficientColumnsForConstraint {
752            found: centers.nrows(),
753        });
754    }
755    validate_chart_points(centers.view(), spec.kappa, "centers")?;
756    // ONE kernel, ONE range (gam#2747). The design and the penalty are the two
757    // blocks of the SAME Gram — `X = k_{κ,ℓ}(data,C)z` and
758    // `S = zᵀk_{κ,ℓ}(C,C)z` — which is what makes `S` the RKHS roughness of the
759    // function `X` realizes, and the whole model the ordinary
760    // subset-of-regressors GP with kernel `exp(−d_κ/ℓ)`, written in the
761    // contrast gauge `k = ℓ·(e^{−d_κ/ℓ} − 1)` that
762    // `constant_curvature_kernel_matrix` derives.
763    //
764    // Two earlier constructions evaluated them at DIFFERENT lengths: `#944`'s
765    // fill-invariant `L(κ)` for the design and `#1464`'s `L_S(κ)` for the
766    // penalty, each solved from its own fill target. Both were attempts to
767    // remove the κ/ℓ confounding by CONSTRAINT — pinning one scalar summary of
768    // the design so κ could not buy resolution — and both fail for the same
769    // reason: pinning a summary selects a one-dimensional curve through the
770    // `(κ, ℓ)` plane a priori, and on such a curve `dV/dκ = V_κ + V_ℓ·L′(κ)`
771    // carries a range term that vanishes only if the heuristic `ℓ_ref` was
772    // already optimal. Measured (gam#2747, truths planted inside the fitted
773    // span): with `ℓ` pinned the criterion recovers κ⋆ only when the truth's own
774    // range IS `ℓ_ref`, and at half or twice that range it rails, inverts the
775    // sign, or invents `κ̂ = ∓0.94` from flat data. The confounding is removed
776    // by ESTIMATING the range instead — `η = ln ℓ` is the smooth's second outer
777    // coordinate — after which the envelope theorem kills the leak exactly.
778    let length_scale = realized_constant_curvature_length_scale(centers.view(), spec.length_scale)?;
779    let raw_penalty =
780        constant_curvature_kernel_matrix(centers.view(), centers.view(), spec.kappa, length_scale)?;
781    // Realized-design constraint transform: uniform coefficient sum-to-zero at
782    // fit time; the frozen composed `z · z_parametric` at predict time (#532
783    // pattern — see ConstantCurvatureIdentifiability).
784    let z = match &spec.identifiability {
785        ConstantCurvatureIdentifiability::FrozenTransform { transform } => {
786            if transform.nrows() != centers.nrows() {
787                crate::bail_dim_basis!(
788                    "frozen constant-curvature identifiability transform mismatch: {} centers but transform has {} rows",
789                    centers.nrows(),
790                    transform.nrows()
791                );
792            }
793            transform.clone()
794        }
795        ConstantCurvatureIdentifiability::CenterSumToZero => {
796            let weights = Array1::<f64>::ones(centers.nrows());
797            weighted_coefficient_sum_to_zero_transform(weights.view())?
798        }
799    };
800    let gauge = gam_problem::Gauge::from_block_transforms(&[z.clone()]);
801    // The penalty is assembled AFTER the gauge, not before it (gam#2747). In the
802    // `exp(−d/ℓ)` gauge the raw `m × m` Gram was itself PSD, so it could be
803    // wrapped first and restricted second; the normalized kernel `ℓ·(e^{−d/ℓ}−1)`
804    // is conditionally negative definite instead — `−k` is CND on every constant
805    // -curvature space form — and is PSD only ON the sum-to-zero frame, where
806    // `zᵀkz = ℓ·zᵀe^{−d/ℓ}z ≻ 0` exactly. Restricting first is therefore not a
807    // convenience: it is where the quadratic becomes a penalty. It is also where
808    // the cancellation the gauge exists to avoid would otherwise reappear —
809    // `zᵀ(ℓK)z` formed from entries that all approach `ℓ` loses the same digits
810    // `Kz` does, while `zᵀkz` formed from `expm1` loses none.
811    let penalty = ConstructiveQuadratic::try_from_dense_psd(
812        symmetrize(&gauge.restrict_penalty(&raw_penalty)),
813        "constant-curvature restricted RKHS penalty",
814    )?;
815    let raw_design =
816        constant_curvature_kernel_matrix(data, centers.view(), spec.kappa, length_scale)?;
817    let design = gam_linalg::matrix::DesignMatrix::Dense(
818        gam_linalg::matrix::DenseDesignMatrix::from(gauge.restrict_design(&raw_design)),
819    );
820    // Keep the RKHS penalty RAW (the symmetric kernel Gram zᵀKz) with
821    // normalization_scale = 1, rather than Frobenius-normalizing it. The Gram's
822    // eigenvalues ARE the physical RKHS roughness energies of each coefficient
823    // direction: the smoothest functions (the low-degree / degree-1 signal) sit
824    // in the genuinely tiny-eigenvalue directions, while wiggly functions sit in
825    // the large ones — a spread of many orders of magnitude. Frobenius-
826    // normalizing divides the whole operator by ‖·‖_F (dominated by the large
827    // wiggly eigenvalues), which compresses that spread and inflates the
828    // smallest eigenvalues relative to their natural scale. REML's scale-
829    // sensitive λ heuristics then drive a single λ high enough to suppress the
830    // wiggly directions and, because the smooth directions are no longer
831    // proportionally tiny, over-shrink the recoverable low-degree signal
832    // (planted degree-1 sphere harmonic recovered at only R²≈0.84). Keeping the
833    // raw physical operator (scale = 1, matching the sphere-harmonic Laplace-
834    // Beltrami penalty) lets REML act on true roughness, leaving the smooth
835    // signal essentially unpenalized while still shrinking the wiggly tail —
836    // raising recovery toward the unconstrained RKHS ceiling. The penalty stays
837    // exactly proportional to zᵀKz, so the constrained-kernel-Gram contract is
838    // unchanged.
839    let mut candidates = vec![PenaltyCandidate {
840        matrix: penalty,
841        source: PenaltySource::Primary,
842        normalization_scale: 1.0,
843        kronecker_factors: None,
844        op: None,
845    }];
846    if spec.double_penalty {
847        // #1531: the primary here is the RKHS kernel Gram zᵀKz, which is
848        // strictly PD / full-rank on distinct centers. It has no unpenalized
849        // function subspace, so an explicit second shrinkage coordinate must
850        // target the whole coefficient chart. The full identity is therefore
851        // intentional for this basis rather than a null-space penalty.
852        // The regression test `constant_curvature_gram_is_full_rank_so_identity_is_the_only_double_penalty`
853        // locks the full-rank fact that justifies this branch.
854        let ridge = Array2::<f64>::eye(design.ncols());
855        let (ridge_norm, c_ridge) = normalize_penalty(&ridge);
856        candidates.push(PenaltyCandidate {
857            matrix: ConstructiveQuadratic::try_from_dense_psd(
858                ridge_norm,
859                "constant-curvature whole-function ridge",
860            )?,
861            source: PenaltySource::DoublePenaltyNullspace,
862            normalization_scale: c_ridge,
863            kronecker_factors: None,
864            op: None,
865        });
866    }
867    let filtered = filter_penalty_candidates(candidates)?;
868    Ok(BasisBuildResult {
869        design,
870        affine_offset: None,
871        active_penalties: filtered.active,
872        dropped_penalties: filtered.dropped,
873        metadata: BasisMetadata::ConstantCurvature {
874            centers,
875            kappa: spec.kappa,
876            length_scale,
877            constraint_transform: Some(z),
878        },
879        kronecker_factored: None,
880        joint_null_rotation: None,
881    })
882}
883
884/// Select constant-curvature centers.
885///
886/// Upper bound on `max‖c‖²`, the largest squared chart radius among the centers
887/// `select_constant_curvature_centers` will return for `strategy` on `data` —
888/// computed WITHOUT materializing them.
889///
890/// Every κ bound is denominated in a chart radius, and the set that radius must
891/// be taken over is the one the kernel EVALUATES: `K_κ` calls
892/// `ConstantCurvature::distance(x, c)` for each (data row, center) pair and
893/// `validate_chart_points` checks data **and** centers. Taking the radius over
894/// `data` alone is only correct while every center is inside the data hull
895/// (gam#2716). Two strategies break that:
896///
897/// * [`CenterStrategy::UserProvided`] — left verbatim by
898///   `select_constant_curvature_centers`, so a center may sit at any radius.
899/// * [`CenterStrategy::UniformGrid`] — the Cartesian product of per-axis
900///   linspaces over the data's *bounding box*, so a corner center sits at the
901///   bounding box corner, radius up to `√d·max‖x‖`, outside the hull for `d ≥ 2`.
902///
903/// Every other strategy assigns either a data row verbatim (equal-mass leaves,
904/// farthest point) or a convex combination of data rows (k-means centroids), so
905/// `max‖c‖ ≤ max‖x‖` and this returns exactly the data radius — which is what
906/// makes the κ box bit-identical to its pre-#2716 value on every data-driven
907/// strategy. The origin-snap in `select_constant_curvature_centers` only ever
908/// moves a center toward the origin, so it cannot invalidate an upper bound.
909///
910/// The match is exhaustive rather than wildcarded: a new strategy has to state
911/// its own radius law instead of silently inheriting a wrong one.
912pub fn constant_curvature_center_chart_radius2(
913    data: ArrayView2<'_, f64>,
914    feature_cols: &[usize],
915    strategy: &CenterStrategy,
916) -> f64 {
917    match strategy {
918        CenterStrategy::Auto(inner) => {
919            constant_curvature_center_chart_radius2(data, feature_cols, inner)
920        }
921        CenterStrategy::DuchonSpectral { knots, .. } => {
922            constant_curvature_center_chart_radius2(data, feature_cols, knots)
923        }
924        CenterStrategy::UserProvided(centers) => {
925            let mut max_r2 = 0.0_f64;
926            for row in centers.outer_iter() {
927                let mut r2 = 0.0_f64;
928                for &v in row.iter() {
929                    if v.is_finite() {
930                        r2 += v * v;
931                    }
932                }
933                max_r2 = max_r2.max(r2);
934            }
935            max_r2
936        }
937        CenterStrategy::UniformGrid { .. } => {
938            // The grid spans `[min_c, max_c]` per axis, so the extreme center
939            // radius is realized at the bounding-box corner whose coordinate is
940            // `max(|min_c|, |max_c|)` on every axis simultaneously.
941            let mut corner_r2 = 0.0_f64;
942            for &c in feature_cols.iter() {
943                let mut lo = f64::INFINITY;
944                let mut hi = f64::NEG_INFINITY;
945                for row in data.outer_iter() {
946                    if let Some(&v) = row.get(c)
947                        && v.is_finite()
948                    {
949                        lo = lo.min(v);
950                        hi = hi.max(v);
951                    }
952                }
953                if lo.is_finite() && hi.is_finite() {
954                    let extreme = lo.abs().max(hi.abs());
955                    corner_r2 += extreme * extreme;
956                }
957            }
958            corner_r2
959        }
960        CenterStrategy::EqualMass { .. }
961        | CenterStrategy::EqualMassCovarRepresentative { .. }
962        | CenterStrategy::FarthestPoint { .. }
963        | CenterStrategy::KMeans { .. } => {
964            constant_curvature_data_chart_radius2(data, feature_cols)
965        }
966    }
967}
968
969/// `max‖x‖²` over the term's feature columns of `data`, the other half of the
970/// evaluated pair set. Non-finite coordinates are skipped rather than poisoning
971/// the maximum (the basis build refuses them separately, by row and name).
972pub fn constant_curvature_data_chart_radius2(
973    data: ArrayView2<'_, f64>,
974    feature_cols: &[usize],
975) -> f64 {
976    let mut max_r2 = 0.0_f64;
977    for row in data.outer_iter() {
978        let mut r2 = 0.0_f64;
979        for &c in feature_cols.iter() {
980            if let Some(&v) = row.get(c)
981                && v.is_finite()
982            {
983                r2 += v * v;
984            }
985        }
986        max_r2 = max_r2.max(r2);
987    }
988    max_r2
989}
990
991/// The stereographic constant-curvature chart has a distinguished pole: the
992/// chart origin.  Curvature sign is visible first in the radial geodesic map
993/// from that pole (`2 atan(√κ r)/√κ` versus `2 atanh(√|κ| r)/√|κ|`).  A pure
994/// farthest-point subset can miss the pole on disk-like clouds, leaving the
995/// radial mode to be reconstructed indirectly from boundary centers; then the
996/// positive chart's distance compression becomes a generic interpolation
997/// advantage and the κ profile is sign-blind.  Keep the user's requested center
998/// count, but make data-driven center sets pole-aware by replacing the center
999/// closest to the origin with the exact origin.  User-provided centers are left
1000/// verbatim.
1001fn select_constant_curvature_centers(
1002    data: ArrayView2<'_, f64>,
1003    strategy: &CenterStrategy,
1004) -> Result<Array2<f64>, BasisError> {
1005    let mut centers = select_centers_by_strategy(data, strategy)?;
1006    match strategy {
1007        CenterStrategy::UserProvided(_) => return Ok(centers),
1008        CenterStrategy::Auto(inner) => {
1009            if matches!(inner.as_ref(), CenterStrategy::UserProvided(_)) {
1010                return Ok(centers);
1011            }
1012        }
1013        CenterStrategy::DuchonSpectral { knots, .. } => {
1014            if center_strategy_kind(knots) == CenterStrategyKind::UserProvided {
1015                return Ok(centers);
1016            }
1017        }
1018        // Every data-driven strategy picks its centers from the cloud and can
1019        // therefore miss the chart origin, so all of them get the pole-aware
1020        // replacement below. Enumerated rather than wildcarded so a new
1021        // strategy has to state whether its centers are user-authored.
1022        CenterStrategy::EqualMass { .. }
1023        | CenterStrategy::EqualMassCovarRepresentative { .. }
1024        | CenterStrategy::FarthestPoint { .. }
1025        | CenterStrategy::KMeans { .. }
1026        | CenterStrategy::UniformGrid { .. } => {}
1027    }
1028    if centers.nrows() == 0 || centers.ncols() == 0 {
1029        return Ok(centers);
1030    }
1031    let (closest, _) = centers
1032        .outer_iter()
1033        .enumerate()
1034        .map(|(i, row)| (i, row.dot(&row)))
1035        .min_by(|a, b| a.1.partial_cmp(&b.1).unwrap_or(std::cmp::Ordering::Equal))
1036        .expect("centers has at least one row; the empty case returned above");
1037    for j in 0..centers.ncols() {
1038        centers[(closest, j)] = 0.0;
1039    }
1040    Ok(centers)
1041}
1042
1043/// The REALIZED center set the builder will use for `spec` on `data` — the
1044/// pole-aware selection, not the raw strategy output.
1045///
1046/// Public because the outer ψ machinery has to derive the range seed and the
1047/// range window from the SAME centers the basis is built on: reading them from
1048/// a re-derived plain strategy would put the ψ box and the objective on two
1049/// different geometries.
1050pub fn constant_curvature_realized_centers(
1051    data: ArrayView2<'_, f64>,
1052    spec: &ConstantCurvatureBasisSpec,
1053) -> Result<Array2<f64>, BasisError> {
1054    let centers = select_constant_curvature_centers(data, &spec.center_strategy)?;
1055    if centers.nrows() < 2 {
1056        return Err(BasisError::InsufficientColumnsForConstraint {
1057            found: centers.nrows(),
1058        });
1059    }
1060    Ok(centers)
1061}
1062
1063/// Symmetrize `M` in place to `(M + Mᵀ)/2` (the realized penalty is built from
1064/// the symmetric kernel Gram; the κ-derivative blocks inherit the same exact
1065/// symmetrization the value path applies before normalization).
1066pub(crate) fn symmetrize(m: &Array2<f64>) -> Array2<f64> {
1067    gam_linalg::matrix::symmetrize(m)
1068}
1069
1070/// Map a single primary-penalty κ-derivative onto the active penalty list by
1071/// source — the constant-curvature analogue of the Matérn double-penalty
1072/// derivative selector. The RKHS Gram is the only κ-moving penalty; the
1073/// double-penalty ridge `I` is κ-independent, so its derivative is exactly
1074/// zero. Any other source would mean the basis grew a penalty whose κ-movement
1075/// is unaccounted for, so we refuse loudly rather than silently drop a term.
1076pub(crate) fn active_constant_curvature_penalty_derivatives(
1077    penalties: &[ActivePenalty],
1078    primary_derivative: &Array2<f64>,
1079) -> Result<Vec<Array2<f64>>, BasisError> {
1080    penalties
1081        .iter()
1082        .map(|penalty| match &penalty.info.source {
1083            PenaltySource::Primary => Ok(primary_derivative.clone()),
1084            PenaltySource::DoublePenaltyNullspace => {
1085                Ok(Array2::<f64>::zeros(primary_derivative.raw_dim()))
1086            }
1087            other => Err(BasisError::InvalidInput(format!(
1088                "unexpected constant-curvature penalty source in κ-derivative path: {other:?}"
1089            ))),
1090        })
1091        .collect()
1092}
1093
1094/// Design and penalty jets of the realized constant-curvature smooth in BOTH
1095/// outer coordinates, `ψ = (κ, η)` with `η = ln ℓ` (gam#2747).
1096///
1097/// The realized construction is `X = K(data,C)·z`, `S = symm(zᵀK(C,C)z)` with
1098/// `z` and the centers ψ-FIXED, so every block below is the corresponding
1099/// kernel jet pushed through the same two ψ-fixed linear maps. The RKHS penalty
1100/// ships raw (`normalization_scale = 1`), so no normalization quotient rule
1101/// participates; the double-penalty ridge `I` is ψ-independent and its
1102/// derivative blocks are exactly zero.
1103#[derive(Clone, Debug)]
1104pub struct ConstantCurvaturePsiJets {
1105    /// `∂X/∂κ`.
1106    pub design_kappa: Array2<f64>,
1107    /// `∂X/∂η`.
1108    pub design_eta: Array2<f64>,
1109    /// `∂²X/∂κ²`.
1110    pub design_kappa2: Array2<f64>,
1111    /// `∂²X/∂κ∂η`.
1112    pub design_kappa_eta: Array2<f64>,
1113    /// `∂²X/∂η²`.
1114    pub design_eta2: Array2<f64>,
1115    /// `∂S_m/∂κ`, one per ACTIVE penalty in the realized basis's order.
1116    pub penalties_kappa: Vec<Array2<f64>>,
1117    /// `∂S_m/∂η`.
1118    pub penalties_eta: Vec<Array2<f64>>,
1119    /// `∂²S_m/∂κ²`.
1120    pub penalties_kappa2: Vec<Array2<f64>>,
1121    /// `∂²S_m/∂κ∂η`.
1122    pub penalties_kappa_eta: Vec<Array2<f64>>,
1123    /// `∂²S_m/∂η²`.
1124    pub penalties_eta2: Vec<Array2<f64>>,
1125}
1126
1127/// Full `(κ, η)` derivative bundle for the constant-curvature smooth — the
1128/// outer-channel hook that lets the signed curvature AND the kernel range join
1129/// the REML optimization as two design-moving coordinates.
1130///
1131/// The curvature coordinate is the **raw, signed κ** (NOT `log κ` as for the
1132/// Matérn kernel scale): κ = 0 must be a reachable interior point of the
1133/// `S^d ← ℝ^d → H^d` family, which `log κ` cannot represent. The range
1134/// coordinate IS logarithmic (`η = ln ℓ`), matching every other kernel smooth in
1135/// the tree, because `ℓ > 0` and the criterion's curvature in `ℓ` is scale-free.
1136///
1137/// Mirrors [`build_constant_curvature_basis`] so the realized design and
1138/// penalties whose ψ-derivatives this returns are byte-for-byte the same
1139/// construction the value path produced (same centers, same ℓ, same `z`).
1140pub fn build_constant_curvature_basis_psi_derivatives(
1141    data: ArrayView2<'_, f64>,
1142    spec: &ConstantCurvatureBasisSpec,
1143) -> Result<ConstantCurvaturePsiJets, BasisError> {
1144    if data.ncols() == 0 {
1145        crate::bail_invalid_basis!("constant-curvature smooth needs at least one feature column");
1146    }
1147    if !spec.kappa.is_finite() {
1148        crate::bail_invalid_basis!("constant-curvature smooth needs a finite kappa");
1149    }
1150    validate_chart_points(data, spec.kappa, "data")?;
1151    // Pole-aware centers, IDENTICAL to `build_constant_curvature_basis` (#1464):
1152    // this bundle's whole contract is that the design/penalty whose ψ-derivatives
1153    // it returns are byte-for-byte the SAME construction the value path produced
1154    // (see the doc above). The value builder replaces the near-origin center with
1155    // the exact pole for sign identifiability; if this bundle re-derived plain
1156    // farthest-point centers instead, ∂X/∂ψ would be the derivative of a DIFFERENT
1157    // design than the frozen one the outer criterion is built on, desyncing the
1158    // analytic gradient from the finite difference of the cost.
1159    let centers = select_constant_curvature_centers(data, &spec.center_strategy)?;
1160    if centers.nrows() < 2 {
1161        return Err(BasisError::InsufficientColumnsForConstraint {
1162            found: centers.nrows(),
1163        });
1164    }
1165    validate_chart_points(centers.view(), spec.kappa, "centers")?;
1166    let length_scale = realized_constant_curvature_length_scale(centers.view(), spec.length_scale)?;
1167
1168    // ψ-fixed constraint transform `z`, resolved exactly as the value builder.
1169    let z = match &spec.identifiability {
1170        ConstantCurvatureIdentifiability::FrozenTransform { transform } => {
1171            if transform.nrows() != centers.nrows() {
1172                crate::bail_dim_basis!(
1173                    "frozen constant-curvature identifiability transform mismatch: {} centers but transform has {} rows",
1174                    centers.nrows(),
1175                    transform.nrows()
1176                );
1177            }
1178            transform.clone()
1179        }
1180        ConstantCurvatureIdentifiability::CenterSumToZero => {
1181            let weights = Array1::<f64>::ones(centers.nrows());
1182            weighted_coefficient_sum_to_zero_transform(weights.view())?
1183        }
1184    };
1185    let gauge = gam_problem::Gauge::from_block_transforms(&[z.clone()]);
1186
1187    // Design ψ-jets: X = K(data, centers)·z at the SAME ℓ the value path uses,
1188    // so the derivatives are the kernel ψ-jets right-multiplied by `z`.
1189    let dc = constant_curvature_kernel_psi_jets(data, centers.view(), spec.kappa, length_scale)?;
1190    // Penalty ψ-jets: S = symm(zᵀ K(centers,centers) z) at that same ℓ.
1191    let cc = constant_curvature_kernel_psi_jets(
1192        centers.view(),
1193        centers.view(),
1194        spec.kappa,
1195        length_scale,
1196    )?;
1197
1198    // Align each primary-penalty derivative with the realized active penalty
1199    // list (primary always; ridge only when double_penalty, and ψ-independent).
1200    // Rebuild the realized basis once to read `penaltyinfo`.
1201    let base = build_constant_curvature_basis(data, spec)?;
1202    let penalty_block = |raw: &Array2<f64>| -> Result<Vec<Array2<f64>>, BasisError> {
1203        let restricted = symmetrize(&gauge.restrict_penalty(raw));
1204        active_constant_curvature_penalty_derivatives(&base.active_penalties, &restricted)
1205    };
1206
1207    Ok(ConstantCurvaturePsiJets {
1208        design_kappa: gauge.restrict_design(&dc.d_kappa),
1209        design_eta: gauge.restrict_design(&dc.d_eta),
1210        design_kappa2: gauge.restrict_design(&dc.d_kappa2),
1211        design_kappa_eta: gauge.restrict_design(&dc.d_kappa_eta),
1212        design_eta2: gauge.restrict_design(&dc.d_eta2),
1213        penalties_kappa: penalty_block(&cc.d_kappa)?,
1214        penalties_eta: penalty_block(&cc.d_eta)?,
1215        penalties_kappa2: penalty_block(&cc.d_kappa2)?,
1216        penalties_kappa_eta: penalty_block(&cc.d_kappa_eta)?,
1217        penalties_eta2: penalty_block(&cc.d_eta2)?,
1218    })
1219}
1220
1221/// The κ slice of [`build_constant_curvature_basis_psi_derivatives`], in the
1222/// generic [`BasisPsiDerivativeBundle`] shape the isotropic spatial ψ-channel
1223/// consumes for callers that hold the range fixed.
1224pub fn build_constant_curvature_basis_kappa_derivatives(
1225    data: ArrayView2<'_, f64>,
1226    spec: &ConstantCurvatureBasisSpec,
1227) -> Result<BasisPsiDerivativeBundle, BasisError> {
1228    let jets = build_constant_curvature_basis_psi_derivatives(data, spec)?;
1229    Ok(BasisPsiDerivativeBundle {
1230        first: BasisPsiDerivativeResult {
1231            design_derivative: jets.design_kappa,
1232            penalties_derivative: jets.penalties_kappa,
1233            implicit_operator: None,
1234        },
1235        second: BasisPsiSecondDerivativeResult {
1236            designsecond_derivative: jets.design_kappa2,
1237            penaltiessecond_derivative: jets.penalties_kappa2,
1238            implicit_operator: None,
1239        },
1240        implicit_operator: None,
1241    })
1242}
1243
1244#[cfg(test)]
1245mod tests {
1246    use super::*;
1247    use gam_linalg::faer_ndarray::FaerEigh;
1248
1249    // Diagnostic (#1059 follow-up): show that a κ-FROZEN chart-scale length
1250    // makes the geodesic-exponential kernel COLLAPSE toward the constant
1251    // function as κ grows positive (sphere distances compress), which is the
1252    // degenerate optimum the REML criterion rails to. For a fixed center set we
1253    // print, per κ, the median geodesic distance and the kernel "spread"
1254    // 1 − mean(offdiag K). A collapsing kernel ⇒ spread → 0 as κ ↑.
1255    #[test]
1256    pub(crate) fn kernel_spread_collapses_with_kappa_at_frozen_length_scale() {
1257        // 8 centers in a disk of radius 0.45 (inside every κ∈[-2,2] chart).
1258        let centers = ndarray::array![
1259            [0.10, 0.05],
1260            [-0.20, 0.15],
1261            [0.30, -0.10],
1262            [-0.05, -0.25],
1263            [0.22, 0.20],
1264            [-0.30, -0.05],
1265            [0.05, 0.30],
1266            [-0.15, 0.10],
1267        ];
1268        // Frozen ℓ: the κ=0 chart-scale auto rule (median 2‖Δ‖).
1269        let ell_frozen = realized_constant_curvature_length_scale(centers.view(), 0.0)
1270            .expect("fixture centers span a positive pairwise distance");
1271
1272        let spread = |kappa: f64, ell: f64| -> f64 {
1273            let k = constant_curvature_kernel_matrix(centers.view(), centers.view(), kappa, ell)
1274                .expect("fixture centers are distinct and the length scale is positive");
1275            let m = k.nrows();
1276            let mut s = 0.0;
1277            let mut cnt = 0.0;
1278            for i in 0..m {
1279                for j in 0..m {
1280                    if i != j {
1281                        s += k[(i, j)];
1282                        cnt += 1.0;
1283                    }
1284                }
1285            }
1286            1.0 - s / cnt
1287        };
1288
1289        let s_neg = spread(-2.0, ell_frozen);
1290        let s_zero = spread(0.0, ell_frozen);
1291        let s_pos = spread(2.0, ell_frozen);
1292        eprintln!(
1293            "[κ-collapse] frozen ℓ={ell_frozen:.4}: spread κ=-2 {s_neg:.4} | κ=0 {s_zero:.4} | κ=+2 {s_pos:.4}"
1294        );
1295
1296        // The degenerate signature: positive κ collapses the kernel toward the
1297        // constant (spread shrinks), so the criterion can buy cheap EDF by
1298        // pushing κ up — this is the unidentifiability we are fixing.
1299        assert!(
1300            s_pos < s_zero && s_zero < s_neg,
1301            "expected kernel spread to shrink with κ at frozen ℓ: κ=-2 {s_neg} κ=0 {s_zero} κ=+2 {s_pos}"
1302        );
1303
1304        // Decompose the κ-monotone REML Occam term. The realized penalty is the
1305        // Frobenius-normalized centered Gram S~ = S_raw/‖S_raw‖_F with
1306        // S_raw = symm(zᵀ K z); the REML evidence carries +½ log|S~|_+ over its
1307        // range. Print log det₊(S~) per κ to see whether the penalty-normalization
1308        // Occam term (not just the modest kernel-spread shift) is what rails κ.
1309        let weights = Array1::<f64>::ones(centers.nrows());
1310        let z = weighted_coefficient_sum_to_zero_transform(weights.view())
1311            .expect("fixture weights are positive, so the sum-to-zero transform exists");
1312        let logdet_norm_penalty = |kappa: f64, ell: f64| -> f64 {
1313            let k = constant_curvature_kernel_matrix(centers.view(), centers.view(), kappa, ell)
1314                .expect("fixture centers are distinct and the length scale is positive");
1315            let s_raw = symmetrize(&z.t().dot(&k).dot(&z));
1316            let (s_norm, _c) = normalize_penalty(&s_raw);
1317            let sym = symmetrize(&s_norm);
1318            let (evals, _v) = FaerEigh::eigh(&sym, faer::Side::Lower)
1319                .expect("the fixture Gram is symmetric, so eigh converges");
1320            let max = evals.iter().cloned().fold(0.0_f64, f64::max);
1321            let tol = max * 1e-9;
1322            evals
1323                .iter()
1324                .filter(|&&e| e > tol)
1325                .map(|&e| e.ln())
1326                .sum::<f64>()
1327        };
1328        let l_neg = logdet_norm_penalty(-2.0, ell_frozen);
1329        let l_zero = logdet_norm_penalty(0.0, ell_frozen);
1330        let l_pos = logdet_norm_penalty(2.0, ell_frozen);
1331        eprintln!(
1332            "[κ-collapse] log|S~|_+ (frozen ℓ): κ=-2 {l_neg:.4} | κ=0 {l_zero:.4} | κ=+2 {l_pos:.4}"
1333        );
1334
1335        // GEODESIC-SCALED ℓ removes the κ-dependence of the kernel resolution:
1336        // set ℓ(κ) = median geodesic distance d_κ among centers. Then the spread
1337        // should be ~κ-invariant. Print the geodesic-ℓ spread per κ.
1338        let geo_median_ell = |kappa: f64| -> f64 {
1339            let m = centers.nrows();
1340            let manifold = ConstantCurvature::new(centers.ncols(), kappa);
1341            let mut dists = Vec::with_capacity(m * (m - 1) / 2);
1342            for i in 0..m {
1343                for j in (i + 1)..m {
1344                    dists.push(
1345                        manifold
1346                            .distance(centers.row(i), centers.row(j))
1347                            .expect("fixture centers lie on the manifold"),
1348                    );
1349                }
1350            }
1351            dists.sort_by(|a, b| a.partial_cmp(b).expect("pairwise distances are finite"));
1352            dists[dists.len() / 2]
1353        };
1354        let gs_neg = spread(-2.0, geo_median_ell(-2.0));
1355        let gs_zero = spread(0.0, geo_median_ell(0.0));
1356        let gs_pos = spread(2.0, geo_median_ell(2.0));
1357        let gl_neg = logdet_norm_penalty(-2.0, geo_median_ell(-2.0));
1358        let gl_zero = logdet_norm_penalty(0.0, geo_median_ell(0.0));
1359        let gl_pos = logdet_norm_penalty(2.0, geo_median_ell(2.0));
1360        eprintln!(
1361            "[κ-collapse] geodesic ℓ: spread κ=-2 {gs_neg:.4} | κ=0 {gs_zero:.4} | κ=+2 {gs_pos:.4}"
1362        );
1363        eprintln!(
1364            "[κ-collapse] geodesic ℓ: log|S~|_+ κ=-2 {gl_neg:.4} | κ=0 {gl_zero:.4} | κ=+2 {gl_pos:.4}"
1365        );
1366
1367        // CANDIDATE FIX: freeze the Frobenius normalization constant at κ=0 so
1368        // the REML Occam term log|S_λ|_+ carries only the GENUINE roughness
1369        // spectrum log|S_raw(κ)|_+ (minus a κ-independent constant), not the
1370        // spurious −r·log‖S_raw(κ)‖_F leak. Compare:
1371        //   (a) log|S_raw(κ)|_+        (un-normalized, true roughness Occam term)
1372        //   (b) log|S_raw(κ)/c₀|_+     (frozen-c₀ normalization at κ=0)
1373        // Both should be κ-IDENTIFYING (a real interior optimum), not monotone.
1374        let logdet_raw = |kappa: f64, ell: f64, c0: f64| -> f64 {
1375            let k = constant_curvature_kernel_matrix(centers.view(), centers.view(), kappa, ell)
1376                .expect("fixture centers are distinct and the length scale is positive");
1377            let s_raw = symmetrize(&z.t().dot(&k).dot(&z));
1378            let scaled = s_raw.mapv(|v| v / c0);
1379            let (evals, _v) = FaerEigh::eigh(&scaled, faer::Side::Lower)
1380                .expect("the fixture Gram is symmetric, so eigh converges");
1381            let max = evals.iter().cloned().fold(0.0_f64, f64::max);
1382            let tol = max * 1e-9;
1383            evals
1384                .iter()
1385                .filter(|&&e| e > tol)
1386                .map(|&e| e.ln())
1387                .sum::<f64>()
1388        };
1389        // c₀ = ‖S_raw(κ=0)‖_F at frozen ℓ.
1390        let k0 = constant_curvature_kernel_matrix(centers.view(), centers.view(), 0.0, ell_frozen)
1391            .expect("fixture centers are distinct and the length scale is positive");
1392        let s_raw0 = symmetrize(&z.t().dot(&k0).dot(&z));
1393        let c0 = s_raw0.iter().map(|v| v * v).sum::<f64>().sqrt();
1394        let r_neg = logdet_raw(-2.0, ell_frozen, c0);
1395        let r_zero = logdet_raw(0.0, ell_frozen, c0);
1396        let r_pos = logdet_raw(2.0, ell_frozen, c0);
1397        eprintln!(
1398            "[κ-collapse] frozen-c₀ log|S_raw/c₀|_+ (frozen ℓ): κ=-2 {r_neg:.4} | κ=0 {r_zero:.4} | κ=+2 {r_pos:.4}"
1399        );
1400        // Finer grid to see the shape of the un-normalized roughness Occam term.
1401        eprint!("[κ-collapse] frozen-c₀ grid:");
1402        for kk in [-2.0, -1.0, -0.5, 0.0, 0.5, 1.0, 2.0] {
1403            eprint!(" κ={kk}:{:.4}", logdet_raw(kk, ell_frozen, c0));
1404        }
1405        eprintln!();
1406    }
1407
1408    /// 8 data rows + 8 centers inside a disk of radius < 0.5 (valid in every
1409    /// κ ∈ [−3, 3] chart). Data ≠ centers so the data→center scale is nontrivial.
1410    pub(crate) fn oracle_disk_design_centers() -> (Array2<f64>, Array2<f64>) {
1411        let centers = ndarray::array![
1412            [0.10, 0.05],
1413            [-0.20, 0.15],
1414            [0.30, -0.10],
1415            [-0.05, -0.25],
1416            [0.22, 0.20],
1417            [-0.30, -0.05],
1418            [0.05, 0.30],
1419            [-0.15, 0.10],
1420        ];
1421        // Deterministic pseudo-random data on a slightly wider disk.
1422        let mut state = 0x2545_f491_4f6c_dd1d_u64;
1423        let mut next = || {
1424            state ^= state << 13;
1425            state ^= state >> 7;
1426            state ^= state << 17;
1427            // map to (−0.42, 0.42)
1428            ((state >> 11) as f64 / (1u64 << 53) as f64 - 0.5) * 0.84
1429        };
1430        let n = 60usize;
1431        let mut data = Array2::<f64>::zeros((n, 2));
1432        for i in 0..n {
1433            data[(i, 0)] = next();
1434            data[(i, 1)] = next();
1435        }
1436        (data, centers)
1437    }
1438
1439    /// Degenerate geometries must refuse the range box by name rather than
1440    /// returning a collapsed or infinite one: a box is a claim about where the
1441    /// design is evaluable, and there is no such claim to make when every
1442    /// evaluated pair is coincident.
1443    #[test]
1444    pub(crate) fn range_box_refuses_a_degenerate_geometry() {
1445        let coincident = ndarray::array![[0.3_f64, -0.2], [0.3, -0.2]];
1446        let error = constant_curvature_length_scale_bounds(coincident.view(), coincident.view())
1447            .expect_err("a cloud with no positive pairwise distance has no range box");
1448        let message = format!("{error}");
1449        assert!(
1450            message.contains("no") && message.contains("positive"),
1451            "the refusal must name what is missing; got {message}"
1452        );
1453        // TWO distinct points are enough. With one distance both ends are the
1454        // SAME `d`, so the box's width is a property of the FORMAT alone:
1455        // `(d/(2√ε)) / (d/(½ln(1/ε))) = ½ln(1/ε)/(2√ε) ≈ 6.0e8`.
1456        let pair = ndarray::array![[0.0_f64, 0.0], [0.2, 0.0]];
1457        let (lo, hi) = constant_curvature_length_scale_bounds(pair.view(), pair.view())
1458            .expect("one positive pairwise distance is enough");
1459        let format_width = (0.5 * -f64::EPSILON.ln()) / (2.0 * f64::EPSILON.sqrt());
1460        assert!(
1461            lo > 0.0 && (hi / lo - format_width).abs() <= 1.0e-9 * format_width,
1462            "a one-distance geometry's box width is the format's, {format_width}; got \
1463             [{lo}, {hi}] with ratio {}",
1464            hi / lo
1465        );
1466    }
1467
1468    /// Every entry of the `(κ, η)` kernel tower must match a central finite
1469    /// difference of the value (first order) and of the first derivatives
1470    /// (second order), on BOTH branches and across the κ = 0 series/closed-form
1471    /// seam. This is the gate the outer ψ gradient and the stationarity
1472    /// certificate stand on: five blocks, so five FD comparisons, and the cross
1473    /// term is differenced along the OTHER coordinate than the one it names so
1474    /// a symmetric-by-construction bug cannot hide in it.
1475    #[test]
1476    pub(crate) fn kernel_psi_jets_match_central_differences_in_both_coordinates() {
1477        let (data, centers) = oracle_disk_design_centers();
1478        let ell0 = realized_constant_curvature_length_scale(centers.view(), 0.0)
1479            .expect("fixture centers span a positive pairwise distance");
1480        let eta0 = ell0.ln();
1481        let at = |kappa: f64, eta: f64| {
1482            constant_curvature_kernel_psi_jets(data.view(), centers.view(), kappa, eta.exp())
1483                .expect("the fixture disk is inside every probed chart")
1484        };
1485        let rel = |exact: &Array2<f64>, fd: &Array2<f64>| -> f64 {
1486            let mut err = 0.0_f64;
1487            let mut scale = 0.0_f64;
1488            for (&a, &b) in exact.iter().zip(fd.iter()) {
1489                err = err.max((a - b).abs());
1490                scale = scale.max(a.abs()).max(b.abs());
1491            }
1492            err / scale.max(1.0)
1493        };
1494        let h = 1.0e-5_f64;
1495        for &kappa in &[-1.5_f64, -0.5, -1e-7, 0.0, 1e-7, 0.8, 1.7] {
1496            for &eta in &[eta0 - 0.7, eta0, eta0 + 0.7] {
1497                let jets = at(kappa, eta);
1498                let kp = at(kappa + h, eta);
1499                let km = at(kappa - h, eta);
1500                let ep = at(kappa, eta + h);
1501                let em = at(kappa, eta - h);
1502                let central = |plus: &Array2<f64>, minus: &Array2<f64>| -> Array2<f64> {
1503                    (plus - minus) / (2.0 * h)
1504                };
1505                let checks = [
1506                    ("∂K/∂κ", &jets.d_kappa, central(&kp.value, &km.value), 1e-6),
1507                    ("∂K/∂η", &jets.d_eta, central(&ep.value, &em.value), 1e-6),
1508                    (
1509                        "∂²K/∂κ²",
1510                        &jets.d_kappa2,
1511                        central(&kp.d_kappa, &km.d_kappa),
1512                        1e-5,
1513                    ),
1514                    (
1515                        // Differenced along η of ∂K/∂κ — the opposite order from
1516                        // the closed form's derivation, so equality is a real
1517                        // check of the mixed partial rather than a tautology.
1518                        "∂²K/∂κ∂η",
1519                        &jets.d_kappa_eta,
1520                        central(&ep.d_kappa, &em.d_kappa),
1521                        1e-5,
1522                    ),
1523                    ("∂²K/∂η²", &jets.d_eta2, central(&ep.d_eta, &em.d_eta), 1e-5),
1524                ];
1525                for (label, exact, fd, tol) in checks {
1526                    let error = rel(exact, &fd);
1527                    assert!(
1528                        error < tol,
1529                        "κ={kappa} η={eta}: {label} disagrees with its central difference: rel={error:.6e}"
1530                    );
1531                }
1532            }
1533        }
1534    }
1535
1536    /// The κ = 0 build is the plain Euclidean exponential smooth in the doubled
1537    /// chart gauge, and the design and the penalty are two blocks of ONE Gram at
1538    /// ONE range at every κ (gam#2747). Reconstructing either at the metadata's
1539    /// own `length_scale` must reproduce the realized block exactly — which is
1540    /// the property the fill-invariant `L(κ)` / `L_S(κ)` pair broke, and the
1541    /// reason the penalty was not the RKHS roughness of its own design.
1542    #[test]
1543    pub(crate) fn design_and_penalty_are_one_gram_at_one_range() {
1544        let (data, centers) = oracle_disk_design_centers();
1545        for kappa in [-1.2_f64, -0.4, 0.0, 0.4, 1.2] {
1546            let spec = ConstantCurvatureBasisSpec {
1547                center_strategy: CenterStrategy::UserProvided(centers.clone()),
1548                kappa,
1549                length_scale: 1.3,
1550                ..Default::default()
1551            };
1552            let built = build_constant_curvature_basis(data.view(), &spec).expect("build");
1553            let BasisMetadata::ConstantCurvature {
1554                length_scale,
1555                constraint_transform,
1556                ..
1557            } = &built.metadata
1558            else {
1559                panic!("expected ConstantCurvature metadata");
1560            };
1561            assert_eq!(
1562                *length_scale, 1.3,
1563                "the realized range is the spec's range, not a κ-remapped one"
1564            );
1565            let z = constraint_transform.as_ref().expect("constraint transform");
1566            let k_dc =
1567                constant_curvature_kernel_matrix(data.view(), centers.view(), kappa, *length_scale)
1568                    .expect("design kernel");
1569            let k_cc = constant_curvature_kernel_matrix(
1570                centers.view(),
1571                centers.view(),
1572                kappa,
1573                *length_scale,
1574            )
1575            .expect("penalty kernel");
1576            let design = built.design.to_dense();
1577            for (a, b) in design.iter().zip(k_dc.dot(z).iter()) {
1578                assert!(
1579                    (a - b).abs() < 1e-12,
1580                    "κ={kappa}: design != K(ℓ)·z ({a} vs {b})"
1581                );
1582            }
1583            let gram = symmetrize(&z.t().dot(&k_cc).dot(z));
1584            let primary = built
1585                .active_penalties
1586                .iter()
1587                .find(|penalty| matches!(penalty.info.source, PenaltySource::Primary))
1588                .expect("primary RKHS penalty");
1589            for (a, b) in gram.iter().zip(primary.matrix.iter()) {
1590                assert!(
1591                    (a - b).abs() < 1e-12,
1592                    "κ={kappa}: penalty != zᵀK(ℓ)z at the SAME ℓ ({a} vs {b})"
1593                );
1594            }
1595        }
1596    }
1597
1598    /// The range box is a CONDITIONING wall, not a statistical one: it must
1599    /// contain the coarse end of the geometry's scale span with room above,
1600    /// because the criterion's own minimum provably leaves that span
1601    /// (gam#2747). And it must sit where the GRAM stops being resolvable, not
1602    /// where the kernel stops being representable — the criterion evaluates a
1603    /// Cholesky of `H = XᵀX + λS`, and the Gram squares the design's dynamic
1604    /// range.
1605    #[test]
1606    pub(crate) fn range_box_is_the_gram_conditioning_wall_and_contains_the_scale_span() {
1607        let (data, centers) = oracle_disk_design_centers();
1608        let (span_lo, span_hi) =
1609            constant_curvature_evaluated_scale_span(data.view(), centers.view())
1610                .expect("the fixture carries positive evaluated distances");
1611        let (lo, hi) = constant_curvature_length_scale_bounds(data.view(), centers.view())
1612            .expect("box is derivable");
1613        let seed = realized_constant_curvature_length_scale(centers.view(), 0.0).expect("seed");
1614        assert!(
1615            span_hi < hi && lo < seed && seed < hi,
1616            "the box [{lo}, {hi}] must contain the coarse end of the evaluated span \
1617             [{span_lo}, {span_hi}] and the auto seed {seed}"
1618        );
1619        // AT `lo` the Gram's dynamic range is exactly one ε — the last one a
1620        // double-precision Cholesky can resolve; half an ℓ below, the Gram's
1621        // far entries have rounded into the diagonal.
1622        // "Resolvable" stated operationally, so the boundary case cannot turn
1623        // on which way a comparison against ε rounds: the quantity must still
1624        // PERTURB 1, and must stop perturbing it just outside the wall.
1625        let gram_range = |ell: f64| (-2.0 * span_hi / ell).exp();
1626        assert!(
1627            1.0 + gram_range(lo) != 1.0,
1628            "at ℓ_lo the Gram's far entries must still perturb the diagonal; got {}",
1629            gram_range(lo)
1630        );
1631        assert!(
1632            1.0 + gram_range(lo / 2.0) == 1.0,
1633            "half an ℓ_lo below, the Gram's far entries must round into the diagonal"
1634        );
1635        // AT `hi` the far end is a MODEL statement, not a numerical one
1636        // (gam#2747): the widest evaluated pair's kernel must have come within
1637        // `√ε` of its `ℓ → ∞` limit `−d`, and must NOT yet be that close an
1638        // order of magnitude below. Truncating the chart there is what makes an
1639        // estimate at the top mean "the kernel is the geodesic distance"
1640        // instead of "the box ended here".
1641        let limit_departure = |ell: f64| {
1642            let k = constant_curvature_kernel_scalar(span_hi, ell);
1643            ((k + span_hi) / span_hi).abs()
1644        };
1645        let root_eps = f64::EPSILON.sqrt();
1646        assert!(
1647            limit_departure(hi) <= root_eps * 1.01,
1648            "at ℓ_hi the widest evaluated pair must BE its own limit to √ε; departure {:.3e} \
1649             against {root_eps:.3e}",
1650            limit_departure(hi)
1651        );
1652        assert!(
1653            limit_departure(hi / 10.0) > root_eps,
1654            "an order of magnitude below ℓ_hi the limit must still be resolvable, or the \
1655             chart is being truncated earlier than the model justifies; departure {:.3e}",
1656            limit_departure(hi / 10.0)
1657        );
1658        // And the old top must be gone: `d_min/√ε` is where the RETIRED `exp`
1659        // gauge lost the design to cancellation, which is a property of a gauge
1660        // this basis no longer uses.
1661        let retired_top = span_lo / root_eps;
1662        assert!(
1663            (hi - retired_top).abs() > 0.1 * retired_top,
1664            "ℓ_hi must no longer be the retired cancellation wall {retired_top:.4e}; got {hi:.4e}"
1665        );
1666    }
1667
1668    #[test]
1669    fn constant_curvature_gram_is_full_rank_so_identity_is_the_only_double_penalty() {
1670        // Centers inside every κ chart, several curvatures spanning sign.
1671        let centers = ndarray::array![
1672            [0.10, 0.05],
1673            [-0.20, 0.15],
1674            [0.30, -0.10],
1675            [-0.05, -0.25],
1676            [0.22, 0.20],
1677            [-0.30, -0.05],
1678            [0.05, 0.30],
1679            [-0.15, 0.10],
1680        ];
1681        let weights = Array1::<f64>::ones(centers.nrows());
1682        let z = weighted_coefficient_sum_to_zero_transform(weights.view())
1683            .expect("fixture weights are positive, so the sum-to-zero transform exists");
1684        // Frozen auto length scale (the κ=0 chart-scale rule; 0.0 ⇒ auto), reused
1685        // across κ so the full-rank check is on the same resolution the basis uses.
1686        let ell = realized_constant_curvature_length_scale(centers.view(), 0.0)
1687            .expect("fixture centers span a positive pairwise distance");
1688
1689        for &kappa in &[-2.0_f64, -0.5, 0.0, 0.5, 2.0] {
1690            let k = constant_curvature_kernel_matrix(centers.view(), centers.view(), kappa, ell)
1691                .expect("fixture centers are distinct and the length scale is positive");
1692            // Primary penalty exactly as the basis builder forms it: symmetrized
1693            // gauge-restricted kernel Gram.
1694            let raw = symmetrize(&z.t().dot(&k).dot(&z));
1695
1696            // (a) The primary is full-rank PD: smallest eigenvalue is strictly
1697            // positive (well above the spectral tolerance), so there is no null
1698            // space for a Marra-Wood ridge to shrink.
1699            let (evals, _v) = FaerEigh::eigh(&raw, faer::Side::Lower)
1700                .expect("the fixture Gram is symmetric, so eigh converges");
1701            let max = evals.iter().cloned().fold(0.0_f64, f64::max);
1702            let min = evals.iter().cloned().fold(f64::INFINITY, f64::min);
1703            assert!(
1704                max > 0.0 && min > max * 1e-9,
1705                "constant-curvature Gram must be full-rank PD at κ={kappa}: \
1706                 min eig {min:e}, max eig {max:e}"
1707            );
1708        }
1709    }
1710
1711    /// The geodesic-exponential kernel is Matérn-½, so it has a CUSP at every
1712    /// center — and that is what blocks a constant-curvature SAE ATOM.
1713    ///
1714    /// The distinction is between the two ways a basis gets used. A GAM smooth
1715    /// evaluates its design at FIXED data rows and never differentiates with
1716    /// respect to the input, so a cusp in `x` is invisible to it and this kernel
1717    /// is exactly right there. A SAE atom's latent coordinate is a FITTED
1718    /// parameter: its solve consumes `∂Φ/∂t` and `∂²Φ/∂t²` (the
1719    /// `SaeBasisSecondJet` contract feeding the Newton/Schur assembly). At a
1720    /// center the first derivative is direction-dependent and the second is
1721    /// unbounded, so a Newton step there has no curvature to trust.
1722    ///
1723    /// This measures it rather than citing it: the central second difference of
1724    /// `K` along a fixed direction through a center is compared against the same
1725    /// quantity a smooth distance away. If the kernel were `C²` both would
1726    /// converge; instead the at-center value grows like `1/h` while the offset
1727    /// one converges, and the test asserts a decade of separation.
1728    ///
1729    /// The kernel is not at fault. Geodesic distance is conditionally negative
1730    /// definite on all three space forms, so `exp(−c·d_κ)` is PD for EVERY κ —
1731    /// which is precisely why it was chosen, and smoother radial families lose
1732    /// that guarantee on spheres. The obstruction is real and belongs to the
1733    /// atom design, not to this module.
1734    #[test]
1735    fn geodesic_exponential_kernel_has_unbounded_curvature_at_a_center() {
1736        let kappa = 0.0_f64;
1737        let ell = 1.0_f64;
1738        let center = ndarray::arr2(&[[0.0_f64, 0.0]]);
1739
1740        // Second difference of `h -> K(center + h·e_x, center)` at the center,
1741        // and at a point a fixed distance away where the kernel is smooth.
1742        let curvature_at = |base: [f64; 2], h: f64| -> f64 {
1743            let probe = ndarray::arr2(&[
1744                [base[0] - h, base[1]],
1745                [base[0], base[1]],
1746                [base[0] + h, base[1]],
1747            ]);
1748            let k = constant_curvature_kernel_matrix(probe.view(), center.view(), kappa, ell)
1749                .expect("fixture centers are distinct and the length scale is positive");
1750            (k[[0, 0]] - 2.0 * k[[1, 0]] + k[[2, 0]]) / (h * h)
1751        };
1752
1753        let mut at_center = Vec::new();
1754        let mut off_center = Vec::new();
1755        for &h in &[1.0e-2_f64, 1.0e-3, 1.0e-4] {
1756            at_center.push(curvature_at([0.0, 0.0], h).abs());
1757            off_center.push(curvature_at([0.5, 0.0], h).abs());
1758        }
1759
1760        // Away from the center the second difference converges: successive
1761        // refinements agree.
1762        assert!(
1763            (off_center[2] - off_center[1]).abs() <= 1.0e-3 * off_center[1].max(1.0),
1764            "the kernel must be C² away from its centers; got {off_center:?}"
1765        );
1766        // At the center it diverges like 1/h: each 10x refinement multiplies it
1767        // by ~10, so three decades of h separate by ~100x.
1768        assert!(
1769            at_center[2] > 10.0 * at_center[0],
1770            "a Matérn-½ cusp must make the second difference diverge as h -> 0; \
1771             got {at_center:?}"
1772        );
1773        assert!(
1774            at_center[2] > 100.0 * off_center[2],
1775            "the at-center curvature must dwarf the smooth-region curvature; \
1776             at-center {:?} vs off-center {:?}",
1777            at_center[2],
1778            off_center[2]
1779        );
1780    }
1781
1782    /// #2458 — the second κ-derivatives shipped WITHOUT a reader.
1783    ///
1784    /// `build_constant_curvature_basis_kappa_derivatives` returns a
1785    /// `BasisPsiDerivativeBundle` whose `.second` carries the κ-second
1786    /// derivatives of the design and of each penalty block. Those are consumed
1787    /// in production (`spatial_optimization.rs` destructures and rotates them
1788    /// for the spatial ψ path), but nothing anywhere pinned their VALUES:
1789    /// grepping for readers of `designsecond_derivative` /
1790    /// `penaltiessecond_derivative` outside the destructuring sites returns
1791    /// nothing. A second derivative that is shipped and consumed but never
1792    /// checked is exactly the input #2458 proposes to build a stationarity
1793    /// CERTIFICATE on, and a wrong certificate is worse than an honestly
1794    /// missing one.
1795    ///
1796    /// This differences the ANALYTIC FIRST derivative, which the κ-gradient
1797    /// path already exercises end to end, so a failure localizes to the
1798    /// second-order construction rather than to the basis itself.
1799    ///
1800    /// The bound is the central-difference error budget, not a tuned number.
1801    /// Truncation is order h^2 times the third derivative and roundoff is
1802    /// order eps times the first derivative over h, so at h = 1e-4 on an
1803    /// order-one chart both sit near 1e-8 relative. Asserting 1e-6 leaves two
1804    /// orders of headroom while still failing a missing or mis-scaled term —
1805    /// which is scale-invariant and would NOT shrink with h, hence the
1806    /// h-halving arm below.
1807    #[test]
1808    fn kappa_second_derivatives_match_a_central_difference_of_the_first_2458() {
1809        let data = ndarray::array![
1810            [0.10, 0.05],
1811            [-0.20, 0.15],
1812            [0.30, -0.10],
1813            [-0.05, -0.25],
1814            [0.22, 0.20],
1815            [-0.30, -0.05],
1816            [0.05, 0.30],
1817            [-0.15, 0.10],
1818        ];
1819        let spec = ConstantCurvatureBasisSpec {
1820            center_strategy: CenterStrategy::FarthestPoint { num_centers: 6 },
1821            ..Default::default()
1822        };
1823        let kappa0 = 0.35_f64;
1824
1825        let first_at = |kappa: f64| {
1826            let mut probe = spec.clone();
1827            probe.kappa = kappa;
1828            let bundle = build_constant_curvature_basis_kappa_derivatives(data.view(), &probe)
1829                .expect("fixture points lie inside the chart for every probed kappa");
1830            (
1831                bundle.first.design_derivative,
1832                bundle.first.penalties_derivative,
1833            )
1834        };
1835
1836        let mut exact_spec = spec.clone();
1837        exact_spec.kappa = kappa0;
1838        let analytic = build_constant_curvature_basis_kappa_derivatives(data.view(), &exact_spec)
1839            .expect("fixture points lie inside the chart at kappa0");
1840        let design_second = analytic.second.designsecond_derivative;
1841        let penalty_second = analytic.second.penaltiessecond_derivative;
1842
1843        let max_rel_error_at = |h: f64| -> (f64, f64) {
1844            let (x_plus, s_plus) = first_at(kappa0 + h);
1845            let (x_minus, s_minus) = first_at(kappa0 - h);
1846
1847            let mut design_error = 0.0_f64;
1848            let mut design_scale = 0.0_f64;
1849            for ((&plus, &minus), &exact) in
1850                x_plus.iter().zip(x_minus.iter()).zip(design_second.iter())
1851            {
1852                let fd = (plus - minus) / (2.0 * h);
1853                design_error = design_error.max((fd - exact).abs());
1854                design_scale = design_scale.max(exact.abs()).max(fd.abs());
1855            }
1856
1857            assert_eq!(
1858                s_plus.len(),
1859                penalty_second.len(),
1860                "penalty block count must not depend on kappa"
1861            );
1862            let mut penalty_error = 0.0_f64;
1863            let mut penalty_scale = 0.0_f64;
1864            for ((block_plus, block_minus), block_exact) in
1865                s_plus.iter().zip(s_minus.iter()).zip(penalty_second.iter())
1866            {
1867                for ((&plus, &minus), &exact) in block_plus
1868                    .iter()
1869                    .zip(block_minus.iter())
1870                    .zip(block_exact.iter())
1871                {
1872                    let fd = (plus - minus) / (2.0 * h);
1873                    penalty_error = penalty_error.max((fd - exact).abs());
1874                    penalty_scale = penalty_scale.max(exact.abs()).max(fd.abs());
1875                }
1876            }
1877            (
1878                design_error / design_scale.max(1.0),
1879                penalty_error / penalty_scale.max(1.0),
1880            )
1881        };
1882
1883        let h = 1.0e-4_f64;
1884        let (design_rel, penalty_rel) = max_rel_error_at(h);
1885        eprintln!(
1886            "[2458-second-fd] h={h:.1e}: design rel={design_rel:.3e} penalty rel={penalty_rel:.3e}"
1887        );
1888        assert!(
1889            design_rel < 1.0e-6,
1890            "d2X/dkappa2 disagrees with a central difference of dX/dkappa: rel={design_rel:.6e}"
1891        );
1892        assert!(
1893            penalty_rel < 1.0e-6,
1894            "d2S/dkappa2 disagrees with a central difference of dS/dkappa: rel={penalty_rel:.6e}"
1895        );
1896
1897        // A MISSING term is scale-invariant: it does not shrink when h does.
1898        // Halving h must not inflate the disagreement, which it would if the
1899        // residual were a genuine missing contribution rather than truncation.
1900        let (design_rel_half, penalty_rel_half) = max_rel_error_at(0.5 * h);
1901        eprintln!(
1902            "[2458-second-fd] h={:.1e}: design rel={design_rel_half:.3e} penalty rel={penalty_rel_half:.3e}",
1903            0.5 * h
1904        );
1905        assert!(
1906            design_rel_half <= design_rel.max(1.0e-9) * 2.0,
1907            "halving h must not inflate the design disagreement (missing-term signature): \
1908             {design_rel:.6e} -> {design_rel_half:.6e}"
1909        );
1910        assert!(
1911            penalty_rel_half <= penalty_rel.max(1.0e-9) * 2.0,
1912            "halving h must not inflate the penalty disagreement (missing-term signature): \
1913             {penalty_rel:.6e} -> {penalty_rel_half:.6e}"
1914        );
1915    }
1916
1917    /// The contrast gauge is the SAME model as `exp(−d_κ/ℓ)` — and the `exp`
1918    /// gauge stops being able to express it (gam#2747).
1919    ///
1920    /// The realized blocks are only ever `K z` and `zᵀ K z`, and `z` annihilates
1921    /// constants, so `K` and `ℓ·(K − 1)` differ by a per-row constant and a
1922    /// positive scale: the second block must equal `ℓ ×` the first, EXACTLY, in
1923    /// exact arithmetic, at every κ and every ℓ. That equality is the licence
1924    /// for the change of gauge, so it is asserted first and at a tolerance that
1925    /// leaves no room (1e-12 relative, mid-box).
1926    ///
1927    /// It is then asserted to FAIL — by a factor of `10⁵` in relative terms — at
1928    /// the top of the derived box, because that is the finding: the `exp` gauge
1929    /// forms `K − 1` by subtracting two numbers that agree to `log₁₀(ℓ/d)`
1930    /// digits, so past `ℓ ≈ 10⁶` the design it emits is not the model's. Only
1931    /// one of the two is still right there, and the reference direction says
1932    /// which: `expm1` is exact by construction and `exp(x) − 1` is not.
1933    #[test]
1934    fn the_contrast_gauge_is_the_same_model_and_the_exp_gauge_loses_it_2747() {
1935        let (data, centers) = oracle_disk_design_centers();
1936        let manifold = ConstantCurvature::new(2, 0.6);
1937        let exp_gauge_design = |ell: f64| -> Array2<f64> {
1938            let mut raw = Array2::<f64>::zeros((data.nrows(), centers.nrows()));
1939            for i in 0..data.nrows() {
1940                for j in 0..centers.nrows() {
1941                    let d = manifold
1942                        .distance(data.row(i), centers.row(j))
1943                        .expect("fixture disk is inside the κ = 0.6 chart");
1944                    raw[(i, j)] = (-d / ell).exp();
1945                }
1946            }
1947            let weights = Array1::<f64>::ones(centers.nrows());
1948            let z = weighted_coefficient_sum_to_zero_transform(weights.view())
1949                .expect("uniform sum-to-zero frame");
1950            raw.dot(&z).mapv(|value| value * ell)
1951        };
1952        let realized_design = |ell: f64| -> Array2<f64> {
1953            let spec = ConstantCurvatureBasisSpec {
1954                center_strategy: CenterStrategy::UserProvided(centers.clone()),
1955                kappa: 0.6,
1956                length_scale: ell,
1957                ..Default::default()
1958            };
1959            build_constant_curvature_basis(data.view(), &spec)
1960                .expect("build")
1961                .design
1962                .to_dense()
1963        };
1964        let disagreement = |ell: f64| -> f64 {
1965            let a = realized_design(ell);
1966            let b = exp_gauge_design(ell);
1967            let mut num = 0.0_f64;
1968            let mut den = 0.0_f64;
1969            for (&x, &y) in a.iter().zip(b.iter()) {
1970                num += (x - y) * (x - y);
1971                den += x * x;
1972            }
1973            (num / den).sqrt()
1974        };
1975        for ell in [0.25_f64, 1.0, 4.0, 16.0] {
1976            let error = disagreement(ell);
1977            assert!(
1978                error < 1.0e-12,
1979                "the two gauges are one model: at ℓ={ell} they differ by {error:.3e}"
1980            );
1981        }
1982        // At the top of the derived box the `exp` gauge forms `K − 1` from
1983        // entries that agree to `d/ℓ`, so its relative error is `≈ ε·ℓ/d` — a
1984        // DERIVED prediction, bracketed by the geometry's own evaluated scale
1985        // span, not a threshold. The measurement has to sit inside that bracket:
1986        // above it would mean some other error source, below it would mean the
1987        // cancellation is not happening and the finding is wrong.
1988        let (_, ell_hi) = constant_curvature_length_scale_bounds(data.view(), centers.view())
1989            .expect("the fixture geometry has a range box");
1990        let (d_min, d_max) = constant_curvature_evaluated_scale_span(data.view(), centers.view())
1991            .expect("the fixture geometry has an evaluated scale span");
1992        let mildest = f64::EPSILON * ell_hi / d_max;
1993        let worst = f64::EPSILON * ell_hi / d_min;
1994        let at_wall = disagreement(ell_hi);
1995        assert!(
1996            at_wall >= mildest && at_wall <= worst,
1997            "at the box top ℓ={ell_hi:.4e} the `exp` gauge's error must be the cancellation \
1998             `ε·ℓ/d`, bracketed by the evaluated span [{d_min:.4e}, {d_max:.4e}] as \
1999             [{mildest:.3e}, {worst:.3e}]; measured {at_wall:.3e}"
2000        );
2001        assert!(
2002            at_wall > 1.0e3 * disagreement(16.0),
2003            "and it must GROW with the range: {at_wall:.3e} at the box top against \
2004             {:.3e} mid-box",
2005            disagreement(16.0)
2006        );
2007    }
2008
2009    /// The `ℓ → ∞` face of the range coordinate is the geodesic-DISTANCE kernel,
2010    /// reached exactly rather than approached (gam#2747).
2011    ///
2012    /// `−d_κ` is conditionally positive definite on every constant-curvature
2013    /// space form, so `X = −D(data,C)z` and `S = −zᵀD(C,C)z` are an ordinary
2014    /// non-degenerate smooth. In the contrast gauge the realized blocks converge
2015    /// to it — with the `1/ℓ` collapse gone there is something LEFT to converge
2016    /// to — and the penalty stays strictly positive definite all the way, which
2017    /// is what makes the limit a point of the parameter space rather than a
2018    /// wall. `20bde053f` reverted the free-range enrollment for want of "a
2019    /// derived stopping rule for a criterion that converges rather than turning
2020    /// over"; a convergent criterion whose limit is a MODEL does not need one.
2021    #[test]
2022    fn the_range_limit_is_the_geodesic_distance_kernel_2747() {
2023        let (data, centers) = oracle_disk_design_centers();
2024        let weights = Array1::<f64>::ones(centers.nrows());
2025        let z = weighted_coefficient_sum_to_zero_transform(weights.view())
2026            .expect("uniform sum-to-zero frame");
2027        for kappa in [-1.1_f64, 0.0, 0.9] {
2028            let manifold = ConstantCurvature::new(2, kappa);
2029            let distances = |a: ArrayView2<'_, f64>, b: ArrayView2<'_, f64>| -> Array2<f64> {
2030                let mut out = Array2::<f64>::zeros((a.nrows(), b.nrows()));
2031                for i in 0..a.nrows() {
2032                    for j in 0..b.nrows() {
2033                        out[(i, j)] = manifold
2034                            .distance(a.row(i), b.row(j))
2035                            .expect("fixture disk is inside every probed chart");
2036                    }
2037                }
2038                out
2039            };
2040            let limit_design = distances(data.view(), centers.view())
2041                .mapv(|d| -d)
2042                .dot(&z);
2043            let limit_penalty = symmetrize(
2044                &z.t()
2045                    .dot(&distances(centers.view(), centers.view()).mapv(|d| -d))
2046                    .dot(&z),
2047            );
2048            let mut previous = f64::INFINITY;
2049            for ell in [1.0e3_f64, 1.0e5, 1.0e7, 1.0e9] {
2050                let spec = ConstantCurvatureBasisSpec {
2051                    center_strategy: CenterStrategy::UserProvided(centers.clone()),
2052                    kappa,
2053                    length_scale: ell,
2054                    ..Default::default()
2055                };
2056                let built = build_constant_curvature_basis(data.view(), &spec).expect("build");
2057                let design = built.design.to_dense();
2058                let penalty = built.active_penalties[0].matrix.clone();
2059                let gap = |a: &Array2<f64>, b: &Array2<f64>| -> f64 {
2060                    let mut num = 0.0_f64;
2061                    let mut den = 0.0_f64;
2062                    for (&x, &y) in a.iter().zip(b.iter()) {
2063                        num += (x - y) * (x - y);
2064                        den += y * y;
2065                    }
2066                    (num / den).sqrt()
2067                };
2068                let design_gap = gap(&design, &limit_design);
2069                let penalty_gap = gap(&penalty, &limit_penalty);
2070                // First order in `1/ℓ`, so each two decades must buy two orders.
2071                assert!(
2072                    design_gap < previous / 50.0,
2073                    "κ={kappa}: the design must converge to the distance kernel; \
2074                     at ℓ={ell:.0e} the gap is {design_gap:.3e} against {previous:.3e} before"
2075                );
2076                previous = design_gap;
2077                assert!(
2078                    penalty_gap < 1.0e-2,
2079                    "κ={kappa}: the penalty must converge too; gap {penalty_gap:.3e} at ℓ={ell:.0e}"
2080                );
2081                let (evals, _) = FaerEigh::eigh(&penalty, faer::Side::Lower).expect("penalty spectrum");
2082                let smallest = evals.iter().cloned().fold(f64::INFINITY, f64::min);
2083                let largest = evals.iter().cloned().fold(0.0_f64, f64::max);
2084                assert!(
2085                    smallest > 1.0e-10 * largest,
2086                    "κ={kappa}: the restricted Gram must stay strictly PD at ℓ={ell:.0e}; \
2087                     spectrum spans [{smallest:.3e}, {largest:.3e}]"
2088                );
2089            }
2090            assert!(
2091                previous < 1.0e-8,
2092                "κ={kappa}: at ℓ=1e9 the design must BE the distance kernel; gap {previous:.3e}"
2093            );
2094        }
2095    }
2096}