Skip to main content

gam_terms/basis/
measure_jet_smooth.rs

1//! Measure-jet spline smooth: multiscale local-jet-residual energy of the
2//! empirical measure (center-quadratured current implementation).
3//!
4//! The term penalizes, at every quadrature point and every scale, the failure
5//! of `f` to be locally affine *in the measure*:
6//!
7//! ```text
8//!   Q = Σ_ℓ  w_ℓ · Σ_i  mass_i · q_i(ε_ℓ)^(1−2α) · R_{i,ℓ},
9//!   w_ℓ = log_step · ε_ℓ^(−η),   η = 2s + d(2−2α),
10//! ```
11//!
12//! where `R_{i,ℓ}` is the residual quadratic form of the exact weighted
13//! local affine projection at center `i` and scale `ε_ℓ`: kernel weights
14//! `w_j = mass_j · exp(−d_ij²/(2ε_ℓ²))`, kernel mass `q_i = Σ_j w_j`, and the
15//! fit `min_b ‖Cv − Φ̃b‖²_W` over weighted-centered values
16//! `Cv = v − (uᵀv)·1` (`u = w/q`) and weighted-centered scaled features
17//! `Φ̃` (rows `(c_j − c_i)/ε`, column means removed under `u`). Rank-deficient
18//! cells use the machine-precision pseudo-inverse of `Φ̃ᵀWΦ̃/q`, so ambient
19//! affine values are projected away exactly instead of paying a ridge toll.
20//!
21//! # Contracts (each is load-bearing; tests pin them)
22//!
23//! - **Exact constant annihilation.** The constant is removed by the weighted
24//!   mean projection `C`, never ridged: `Q·1 = 0` to machine precision at
25//!   every scale, so the penalty carries NO mass term and the fit has no
26//!   prior mean to revert to. This is the no-mean-reversion contract of the
27//!   measure-jet design; ridging the constant would silently reintroduce
28//!   mean reversion.
29//! - **Exact affine projection / rank adaptation.** The slope block uses the
30//!   rank-revealing pseudo-inverse of the dimensionless local Gram
31//!   `G = Φ̃ᵀWΦ̃/q`, not a Tikhonov ridge. On a 1-D filament in ambient
32//!   dimension d the resolved tangent slope is absorbed (not penalized);
33//!   unresolved directions have no variation after weighted centering and do
34//!   not create an affine toll. The retained rank is a numerical property of
35//!   the weighted cell, not a smoothing dial.
36//! - **Mellin band.** Scales form a geometric grid from the center-spacing
37//!   floor to the half-diameter; `w_ℓ = log_step · ε_ℓ^(−η)`, with
38//!   `η = 2s + d(2−2α)`, is the fixed-order quadrature weight used by this
39//!   implementation. It keeps the advertised continuous smoothness order
40//!   `s ∈ (0, 2)` from silently changing when `α` changes.
41//! - **Density normalization.** The outer quadrature weight
42//!   `mass_i · q_i^(1−2α)` realizes `dμ(x)/q_ε(x)^(2α−1)`. On a p-dimensional
43//!   stratum with sampling density `ρ`, `q_ε ~ Cρ ε^p` and the local residual
44//!   contributes an extra `ε^{p(2−2α)}` factor. The fixed-order scale weight
45//!   cancels that factor using the available dimension parameter; without that
46//!   correction, the symbol exponent would be `2s + 2p(α−1)`.
47//! - **Frozen-quadrature replay.** The penalty and extrapolation diagnostic
48//!   depend on the FIT data through center masses, the realized band, on-web
49//!   support anchors, and penalty normalization scales. The freeze step
50//!   persists all of them ([`MeasureJetFrozenQuadrature`]) so predict-time
51//!   rebuilds replay the exact fit-time penalty instead of recomputing it from
52//!   predict rows.
53//! - **Single assembly source.** Every quadratic form this module emits —
54//!   the energy, its (s, α) jets, the per-scale spectrum — is produced by
55//!   ONE workhorse ([`assemble_weighted_forms`]) that walks the local
56//!   residual blocks exactly once per request and differs only in the
57//!   scalar weights applied per block. Criterion value and criterion
58//!   derivatives cannot drift apart (the objective↔gradient desync class is
59//!   structurally excluded).
60//! - **single-scale/multiscale opt-in (#1039/#1116).** The per-scale spectrum
61//!   and the `(α, ln τ)` ψ dials are the multiscale-mode realization, engaged
62//!   ONLY when the spec opts in (`MeasureJetBasisSpec::multiscale = true`, the
63//!   DSL `mjs(…, multiscale=true)`); see [`measure_jet_multiscale_mode`]. There
64//!   is NO center-count auto-gate: at ANY center count the default is
65//!   single-scale — one jet-energy Primary at the auto order with no energy ψ
66//!   dials. The independent function-space null-component candidate requested
67//!   by `double_penalty` is present in either mode and has its own REML λ. The
68//!   flag is persisted on the spec, so freeze→replay re-enters the same mode
69//!   verbatim.
70//!
71//! # ψ-differentiability contract (what the ψ-channel stage consumes)
72//!
73//! Mirroring the constant-curvature κ-contract (#944): centers, masses, the
74//! band are deliberately hyperparameter-FIXED at build time; the representer
75//! range ℓ is the ONE design-moving dial, and it is REML-selected by default
76//! (#1116, restored in #2761) — λ shrinks inside a span and cannot move one, so
77//! the range that decides WHICH span the representers occupy has to be chosen
78//! by the criterion, exactly as the Matérn κ is. Consequences:
79//!
80//! - **Penalty-dial design drift is identically zero**: the (s, α, τ) dials
81//!   reweight only the jet-energy penalty, never the Gaussian representer
82//!   design (`∂X/∂{s,α,τ} ≡ 0`), so those channels are penalty-only
83//!   (`is_penalty_like` auto-derives true in the outer engine's
84//!   `DirectionalHyperParam`).
85//! - **The representer range ℓ is a design-and-pullback-moving dial** (matérn's
86//!   `log_kappa` analog, #1116): `X = K(data, centers; ℓ)·z` and the center
87//!   evaluation map `E = K(centers, centers; ℓ)·z` both depend on ℓ. The
88//!   center-value forms `Q` and `H₀` are ℓ-invariant, but their coefficient
89//!   pullbacks `EᵀQE` and `EᵀH₀E` are not; exact product-rule jets are shipped
90//!   alongside the design jets. ℓ rebuilds the design per outer trial; it does
91//!   not change the frozen basis rank. FD-gated by
92//!   `psi_producer_matches_fd_length_scale`. Frozen only where a design-moving
93//!   kernel scale on covariates SHARED by two coupled blocks is an
94//!   identifiability hazard — the BMS marginal/log-slope pair, at its own entry
95//!   point (`freeze_measure_jet_length_scale_learning`, #1116/`a3afd17a2`) —
96//!   and where the user pins `length_scale=` outright.
97//! - **Exact (s, α) penalty jets are shipped**:
98//!   [`measure_jet_energy_form_with_jets`] returns `∂Q/∂s`, `∂²Q/∂s²`,
99//!   `∂Q/∂α`, `∂²Q/∂α²`, `∂²Q/∂s∂α` in closed form — both dials enter only
100//!   through the per-block log-weights (`∂ln w/∂s = −2 ln ε`,
101//!   `∂ln w/∂α = −2 ln q`), so the jets are reweighted re-scatters of the
102//!   SAME residual blocks, FD-gated in this module's tests.
103//!   The retained τ coordinate is inert under the exact projection, so its
104//!   derivative slots are identically zero.
105//!
106//! # Cost shape (and the upgrade ladder above it)
107//!
108//! The outer sum is coarsened per scale to a deterministic ε/2-net (the
109//! outer Riemann sum needs resolution ε, not the center-spacing floor), so
110//! the band totals ~O(m²·d) instead of O(L·m³) — the current realization of
111//! the pyramid principle that each scale interacts at its own level. This is
112//! mass-lumped quadrature of the displayed outer integral; it is first-
113//! moment exact for the cell locations and carries the usual
114//! `O(diam²/ε²)` relative scale for smooth Gaussian-weighted functionals,
115//! not an estimand-preserving identity.
116//! The long-form home for the ladder and the substrate contracts is the
117//! frame notes (`docs/measure_jet_frame.md`); its §2 moment substrate is
118//! `measure_jet_moments.rs`, its §5 extrapolation pricing
119//! `measure_jet_predict.rs`.
120
121use ndarray::{Array1, Array2, ArrayView1, ArrayView2, Axis};
122use rayon::prelude::*;
123use serde::{Deserialize, Serialize};
124
125use faer::Side;
126
127use gam_linalg::faer_ndarray::{
128    FaerEigh, FaerSvd, default_rrqr_rank_alpha, rrqr_nullspace_basis,
129};
130
131use super::{
132    AnisoBasisPsiDerivatives, AnisoPenaltyCrossProvider, BasisBuildResult, BasisError,
133    BasisMetadata, CenterStrategy, ConstructiveQuadratic, PenaltyCandidate, PenaltySource,
134    filter_penalty_candidates, normalize_penalty, normalize_penalty_cross_psi_derivative,
135    normalize_penaltywith_psi_derivatives, select_centers_by_strategy, trace_of_product,
136};
137
138/// Truncation radius of the Gaussian profile in units of the scale ε: weights
139/// beyond `3ε` are below `e^{-4.5} ≈ 1.1e-2` of the peak and are dropped from
140/// both the local fit and the `q^(1−2α)` outer weight. This is an absolute
141/// kernel-weight cutoff; using the same truncated q keeps the discrete
142/// functional self-consistent, but it is not a relative tail-error bound.
143pub(crate) const MEASURE_JET_PROFILE_CUTOFF: f64 = 3.0;
144
145/// Relative eigenvalue threshold for rank-revealing pseudo-inverses of local
146/// Gram matrices. Directions at the roundoff floor are treated as unresolved
147/// and excluded from the affine fit.
148pub(crate) const MEASURE_JET_PSEUDOINVERSE_RTOL: f64 = 64.0 * f64::EPSILON;
149
150/// Default continuous smoothness order `s` realized by the `0.0` auto
151/// sentinel. Sits mid-band in the admissible `(0, 2)` for the affine-jet
152/// (r = 2) energy: rough enough to stay pointwise-defined on filaments and
153/// sheets (`s > p/2` for intrinsic `p ≤ 2`), smooth enough to bridge gaps
154/// with attested trends.
155pub(crate) const MEASURE_JET_DEFAULT_ORDER_S: f64 = 1.5;
156
157/// Auto-band scale-count clamp: at least 3 octave-ish nodes so the energy is
158/// genuinely multiscale, at most 8 so degenerate spacing cannot explode the
159/// build.
160pub(crate) const MEASURE_JET_MIN_AUTO_SCALES: usize = 3;
161pub(crate) const MEASURE_JET_MAX_AUTO_SCALES: usize = 8;
162
163/// Representer-range multiple of the median nearest-center spacing used by the
164/// `0.0` auto sentinel.
165///
166/// **This is the SEED of the ℓ outer coordinate, not the realized range.**
167/// [`MeasureJetBasisSpec::learn_length_scale`] is on by default, so what a fit
168/// ships is whatever REML certifies from here; this constant only has to put
169/// the optimizer somewhere feasible and well-conditioned. It is a starting
170/// point in exactly the sense the Matérn `MaternLengthScale::Auto` seed is one.
171///
172/// Set to ×1: a Gaussian representer of range `ℓ = h` (the median
173/// nearest-center spacing) already overlaps its neighbors at
174/// `exp(−h²/(2ℓ²)) = exp(−1/2) ≈ 0.61`, so adjacent bumps blend smoothly while
175/// each center keeps a *distinct* response and the design Gram is far from
176/// rank-deficient. The old ×2 seed made every column `exp(−1/8) ≈ 0.88` at its
177/// neighbor, driving the Gram toward rank deficiency so the inner PIRLS / outer
178/// REML conditioning degraded and the search cycled for hundreds of seconds
179/// (the #1116 timeout). ×1 is the resolving, well-conditioned end of the range
180/// axis — the right side to *start* a search from.
181///
182/// It is emphatically NOT the right place to *stop*. #1041 changed the factor
183/// ×2 → ×1 as a replacement for the ℓ dial it was turning off, on the argument
184/// that a spacing-width kernel "fixes both at the root". Measured on
185/// `measure_jet_perf_parity` at ×1, the design's least-squares span floor
186/// against the noiseless truth is `0.152` and REML moves ℓ 7.5× longer to a
187/// floor of `1.4e-5` (#2761). No fixed multiple of the center spacing is the
188/// answer, because the answer depends on the target's smoothness relative to
189/// the center layout — which is data, not geometry.
190pub(crate) const MEASURE_JET_AUTO_LENGTH_SCALE_FACTOR: f64 = 1.0;
191
192/// Memory budget (in f64 entries) above which the multi-form assembly stops
193/// parallelizing over scales: parallel scale partials cost
194/// `L · n_forms · m²` doubles; past this budget the scales run sequentially
195/// (same numbers — the per-scale loop and the ordered sum are deterministic
196/// either way).
197pub(crate) const MEASURE_JET_PARALLEL_FORM_BUDGET_DOUBLES: usize = 1 << 26;
198
199/// Realized-design identifiability policy for the measure-jet smooth.
200/// Mirrors [`super::ConstantCurvatureIdentifiability`] (#532): the fit-time
201/// section gets the parametric orthogonalization composed onto it by the global
202/// identifiability pipeline, and the composed transform is frozen so
203/// predict-time (and per-ψ-trial) rebuilds replay it verbatim.
204#[derive(Debug, Clone, Serialize, Deserialize, Default)]
205pub enum MeasureJetIdentifiability {
206    /// Fit-time default. With the single-scale affine head, the RBF center
207    /// values are mass-orthogonalized against the affine value space and the
208    /// head passes through exactly; without a head, the representer coefficient
209    /// sum-to-zero section is used. Global parametric residualization follows.
210    #[default]
211    CenterSumToZero,
212    /// Predict-time replay: the frozen composed transform captured at fit
213    /// time. `transform.nrows()` equals representer count plus affine-head width.
214    FrozenTransform { transform: Array2<f64> },
215}
216
217/// Fit-time quadrature of the empirical measure (center masses + realized
218/// scale band), frozen onto the spec so predict-time rebuilds replay the
219/// exact fit-time penalty. Recomputing either from predict rows would
220/// silently change the penalty the coefficients were estimated under.
221#[derive(Debug, Clone, Serialize, Deserialize)]
222pub struct MeasureJetFrozenQuadrature {
223    /// Per-center masses `m_i` (nearest-center fractions of the FIT rows).
224    pub masses: Array1<f64>,
225    /// Realized geometric scale band `ε_0 < … < ε_{L−1}`.
226    pub eps_band: Vec<f64>,
227    /// Per-scale on-web support anchor
228    /// `q̄_ℓ = (Σ_i m_i q_ℓ(c_i)) / (Σ_i m_i)`.
229    pub support_means: Vec<f64>,
230    /// Frobenius scales of the emitted per-level normalized penalties. Empty in
231    /// fused mode, where the band emits one primary penalty instead.
232    pub penalty_normalization_scales: Vec<f64>,
233    /// Frobenius scales of the raw per-level forms before the arbitrary Mellin
234    /// `log_step · ε_ℓ^(-2s0)` gauge is folded in.
235    pub raw_penalty_normalization_scales: Vec<f64>,
236    /// Frobenius scale of the single fused primary penalty. `None` in per-level
237    /// mode.
238    pub fused_penalty_normalization_scale: Option<f64>,
239    /// Ambient input-measurement-error scale `σ_coord` (issue #2225): the
240    /// perpendicular off-manifold residual spread of the fit-time empirical
241    /// measure, in the frozen centers' (standardized) coordinate frame. Frozen
242    /// so the predict-time errors-in-variables variance term
243    /// `Var_input = σ_coord²·‖∇f̂‖²` uses the same input-noise scale the fit
244    /// saw. `None` when it could not be estimated (no cell spanned a tangent),
245    /// leaving `Var_input` disabled. Defaults to `None` for models persisted
246    /// before the term existed.
247    #[serde(default)]
248    pub sigma_coord: Option<f64>,
249}
250
251/// Serde default for [`MeasureJetBasisSpec::learn_length_scale`]: REML-select
252/// ℓ, the same standing the Matérn κ has. A function (not a literal) because
253/// `#[serde(default)]` on a `bool` deserializes a missing field as `false`,
254/// which would silently freeze ℓ on every spec that predates the field — the
255/// opposite of the default. See the field's own docs for the measurement.
256fn measure_jet_learn_length_scale_default() -> bool {
257    true
258}
259
260/// Measure-jet smooth configuration (`mjs(x0, …, xd)`).
261///
262/// The feature columns are ambient coordinates of data concentrated near an
263/// unknown low-dimensional (possibly stratified) set; the term learns the
264/// geometry from the empirical measure itself — centers as quadrature nodes,
265/// masses as μ-weights, local jet residuals as the roughness carrier — with
266/// no graph, mesh, or neighbor-set inside the statistical object.
267#[derive(Debug, Clone, Serialize, Deserialize)]
268pub struct MeasureJetBasisSpec {
269    /// Center/knot selection strategy (deterministic; quadrature of μ).
270    pub center_strategy: CenterStrategy,
271    /// Continuous smoothness order `s ∈ (0, 2)`; `0.0` sentinel = auto
272    /// (`MEASURE_JET_DEFAULT_ORDER_S`).
273    pub order_s: f64,
274    /// Density-normalization exponent α (outer weight `q^{1−2α}`).
275    pub alpha: f64,
276    /// Historical τ coordinate retained for frozen specs and ψ layout. The
277    /// measure-jet energy itself uses the exact weighted affine projection and
278    /// is independent of τ; the τ ψ derivatives are therefore zero.
279    pub tau0: f64,
280    /// Number of scale nodes; `0` sentinel = auto dyadic band.
281    pub num_scales: usize,
282    /// Representer (Gaussian RBF) range ℓ; `0.0` sentinel = auto
283    /// (median nearest-center spacing × `MEASURE_JET_AUTO_LENGTH_SCALE_FACTOR`).
284    pub length_scale: f64,
285    /// Add a separate function-space affine/null-component penalty alongside
286    /// the jet-energy penalty. Its strength is independently REML-selected.
287    pub double_penalty: bool,
288    /// REML-select the representer range ℓ as a design-moving outer dial,
289    /// mirroring Matérn's `log_kappa`. **Default `true`** — ℓ is a basis
290    /// coordinate of the same kind as the Matérn κ, not a smoothing parameter,
291    /// and a fitted model must choose it.
292    ///
293    /// # Why it cannot be a frozen geometric value (#2761)
294    ///
295    /// The Gaussian kernel is strictly PD for every ℓ > 0, so ℓ does NOT change
296    /// the basis rank (always `m` centers) — but it changes WHICH `m`-dim
297    /// subspace the representers span. λ can only shrink inside a span; it
298    /// cannot move one. So a mis-set ℓ is an error no smoothing parameter can
299    /// repair, and the size of that error is not small. Measured on
300    /// `measure_jet_perf_parity`'s 1-D-curve-in-3-D Gaussian fixture
301    /// (`n = 1500`, σ = 0.10, 16 centers, `p = 15`), where `span floor` is the
302    /// least-squares projection residual of the NOISELESS truth onto the
303    /// realized design's column span — the bound no λ can beat:
304    ///
305    /// ```text
306    ///   arm                       ell      edf   span floor  unpen. LS  held-out
307    ///   frozen (auto ell)      0.5144   14.684    0.152488   0.155484   0.155584
308    ///   REML-selected ell      3.8813   14.006    0.000014   0.008155   0.009642
309    ///   matern(k=16)                -   14.619    0.006077   0.011989   0.011639
310    ///   duchon(k=16)                -   15.016    0.002443   0.011308   0.010521
311    /// ```
312    ///
313    /// At the frozen range the fit is already at `edf/p = 0.98` and its held-out
314    /// RMSE *is* the span floor: unpenalized least squares on the same design
315    /// gives 0.1555, and dropping the null-component penalty moves the fourth
316    /// decimal. Freeing ℓ drops the floor by four orders and the held-out RMSE
317    /// by 16x, past both comparators, at LOWER edf — nothing is traded for it.
318    ///
319    /// # History (so a fourth flip needs new evidence)
320    ///
321    /// `299c83ffc` (#1116) introduced this dial default-ON precisely to remove
322    /// this fixture's 13x. `a3afd17a2` then found the one place it is unsafe —
323    /// a BMS fit shares ONE mjs basis between the marginal mean and the
324    /// log-slope surface, and a design-moving kernel scale on shared covariates
325    /// is an identifiability hazard that reached a separation runaway — and
326    /// contained it AT THE BMS ENTRY POINT with
327    /// [`crate::smooth::freeze_measure_jet_length_scale_learning`], which is
328    /// still what runs there. `b1d94d1a5` (#1041) nevertheless flipped the
329    /// GLOBAL default off, and the 13x returned as #2761. The scoped freeze is
330    /// the correct containment; the global one buys nothing it does not already
331    /// buy and costs every single-surface fit its span alignment.
332    ///
333    /// `false` freezes ℓ at the auto (or explicit) value with no outer
334    /// enrollment. The term builder selects that automatically when the user
335    /// pins `length_scale=` — an explicit range is a request, not a seed —
336    /// mirroring the Matérn `all_spatial_terms_kappa_fixed` short-circuit.
337    #[serde(default = "measure_jet_learn_length_scale_default")]
338    pub learn_length_scale: bool,
339    /// Explicit opt-in for multiscale mode: the per-scale spectral penalty
340    /// split plus the `(α, ln τ)` outer ψ dials. `false` (default) keeps the
341    /// energy in single-scale mode at ANY center count. The separate
342    /// `double_penalty` null component is available in both modes. There is no
343    /// center-count auto-gate; the user opts in via
344    /// `mjs(…, multiscale=true)`. Persisted on the spec so freeze→replay enters
345    /// the same mode.
346    #[serde(default)]
347    pub multiscale: bool,
348    /// Realized-design identifiability policy (see type docs).
349    #[serde(default)]
350    pub identifiability: MeasureJetIdentifiability,
351    /// Fit-time quadrature replay (see type docs). `None` at fit time;
352    /// `Some` on the frozen predict/ψ-trial path.
353    #[serde(default)]
354    pub frozen_quadrature: Option<MeasureJetFrozenQuadrature>,
355}
356
357impl Default for MeasureJetBasisSpec {
358    fn default() -> Self {
359        Self {
360            center_strategy: CenterStrategy::FarthestPoint { num_centers: 50 },
361            order_s: 0.0,
362            // Density-WEIGHTED Hessian energy (the module-header default): the
363            // outer weight is q^{1−2α} = q^{−1} at α = 1. The density-free
364            // variant α = 3/2 gives q^{−2}, which on a low-intrinsic-dimension
365            // stratum (data on a 1-D/2-D manifold embedded in higher ambient d)
366            // makes the local kernel mass q tiny AND spatially varying along
367            // the manifold, so q^{−2} amplifies the penalty unevenly and
368            // over-smooths the high-frequency signal there (MEASURED #1116: on
369            // the 1-D-curve-in-3-D fixture α = 3/2 left mjs ~13× worse than
370            // matérn). α = 1's q^{−1} weighting is far gentler and is the
371            // header-derived default; an explicit `alpha=` still overrides for
372            // genuinely density-free use on a full-dimensional stratum.
373            alpha: 1.0,
374            tau0: 1e-3,
375            num_scales: 0,
376            length_scale: 0.0,
377            double_penalty: true,
378            learn_length_scale: true,
379            multiscale: false,
380            identifiability: MeasureJetIdentifiability::CenterSumToZero,
381            frozen_quadrature: None,
382        }
383    }
384}
385
386/// Realized geometric scale band: `eps` ascending, `log_step` the constant
387/// log-spacing `ln(eps[ℓ+1]/eps[ℓ])` used as the Mellin quadrature weight.
388pub struct MeasureJetBand {
389    pub eps: Vec<f64>,
390    pub log_step: f64,
391}
392
393/// The energy and its exact hyperparameter jets in the live dials. `s` and
394/// `α` enter only through per-block log-weights. The retained `ln τ` slots
395/// are zero because the local fit is the exact weighted affine projection
396/// and no longer depends on τ. All forms are scattered from the SAME local
397/// residual blocks, and the ψ-channel consumes them with zero design drift.
398pub struct MeasureJetEnergyJets {
399    pub q: Array2<f64>,
400    pub dq_ds: Array2<f64>,
401    pub d2q_ds2: Array2<f64>,
402    pub dq_dalpha: Array2<f64>,
403    pub d2q_dalpha2: Array2<f64>,
404    pub d2q_ds_dalpha: Array2<f64>,
405    pub dq_dlogtau: Array2<f64>,
406    pub d2q_dlogtau2: Array2<f64>,
407    pub d2q_ds_dlogtau: Array2<f64>,
408    pub d2q_dalpha_dlogtau: Array2<f64>,
409}
410
411/// Householder vector `u` for the uniform sum-to-zero constraint: the
412/// reflection `H = I − 2uuᵀ` maps `c̄ = 1/√m·1` onto `e₁`, so columns 2..m
413/// of `H` are an orthonormal basis of `1⊥` — the same model space as the
414/// generic RRQR nullspace basis, but with O(rows·m) STRUCTURED application
415/// (`X·z = (X − 2(Xu)uᵀ) minus column 1`) instead of the O(rows·m²)
416/// constraint GEMM that the scale-smoke gate identified as the dominant
417/// build cost.
418pub(crate) fn householder_sum_to_zero_u(m: usize) -> Array1<f64> {
419    let c = 1.0 / (m as f64).sqrt();
420    let mut u = Array1::<f64>::from_elem(m, c);
421    u[0] -= 1.0;
422    let norm = u.dot(&u).sqrt();
423    u.mapv_inplace(|v| v / norm);
424    u
425}
426
427/// Materialize the Householder sum-to-zero basis `z` (m × (m−1)) — columns
428/// 2..m of `H = I − 2uuᵀ` — for the frozen-replay metadata. O(m²), built
429/// once per fit.
430pub(crate) fn householder_sum_to_zero_z(u: &Array1<f64>) -> Array2<f64> {
431    let m = u.len();
432    let mut z = Array2::<f64>::zeros((m, m - 1));
433    for j in 0..(m - 1) {
434        for i in 0..m {
435            let h = if i == j + 1 { 1.0 } else { 0.0 } - 2.0 * u[i] * u[j + 1];
436            z[(i, j)] = h;
437        }
438    }
439    z
440}
441
442pub(crate) fn symmetric_pseudoinverse(
443    a: &Array2<f64>,
444    label: &str,
445) -> Result<Array2<f64>, BasisError> {
446    let n = a.nrows();
447    if a.ncols() != n {
448        crate::bail_dim_basis!(
449            "measure-jet pseudo-inverse `{label}` needs a square matrix, got {:?}",
450            a.dim()
451        );
452    }
453    let (evals, evecs) = a.eigh(Side::Lower).map_err(|e| {
454        BasisError::InvalidInput(format!(
455            "measure-jet pseudo-inverse `{label}` eigendecomposition failed: {e}"
456        ))
457    })?;
458    let lam_max = evals.iter().fold(0.0_f64, |acc, v| acc.max((*v).max(0.0)));
459    let rank_tol = MEASURE_JET_PSEUDOINVERSE_RTOL * (n.max(1) as f64) * lam_max;
460    let mut scaled = evecs.clone();
461    for (k, mut col) in scaled.axis_iter_mut(Axis(1)).enumerate() {
462        let lam = evals[k].max(0.0);
463        let inv = if lam > rank_tol { 1.0 / lam } else { 0.0 };
464        col.mapv_inplace(|v| v * inv);
465    }
466    Ok(scaled.dot(&evecs.t()))
467}
468
469/// Rank-adapted center values of the measure-jet energy's affine null space.
470///
471/// The first column is the constant. The remaining columns are the supported
472/// ambient-linear directions returned by [`measure_jet_affine_head_transform`].
473/// Using that transform makes the basis full-column-rank even when the centers
474/// lie on a lower-dimensional affine stratum of the ambient coordinates.
475fn measure_jet_affine_value_basis(
476    centers: ArrayView2<'_, f64>,
477    masses: ArrayView1<'_, f64>,
478) -> Array2<f64> {
479    // The SAME object the design's head block is built from, evaluated at the
480    // centers. Sharing one construction is what makes "the head spans exactly
481    // the energy's null space" a property of the code rather than a comment
482    // two call sites have to keep agreeing on (#2751).
483    let lift = measure_jet_affine_head_lift(centers, masses);
484    measure_jet_affine_head_block(centers, lift.view())
485}
486
487/// Mass-metric quadratic form selecting the affine/null component of center
488/// function values:
489///
490/// `H₀ = W A (Aᵀ W A)⁺ Aᵀ W`.
491///
492/// This is a function-space object: `vᵀH₀v` is the squared mass norm of the
493/// affine projection of the center values `v`. No coefficient metric enters.
494fn affine_function_nullspace_form(
495    centers: ArrayView2<'_, f64>,
496    masses: ArrayView1<'_, f64>,
497) -> Result<Array2<f64>, BasisError> {
498    let m = centers.nrows();
499    if masses.len() != m {
500        crate::bail_dim_basis!(
501            "measure-jet affine function-space form shape mismatch: centers {:?}, masses {}",
502            centers.dim(),
503            masses.len()
504        );
505    }
506    let affine = measure_jet_affine_value_basis(centers, masses);
507    let mut weighted_affine = affine.clone();
508    for (i, mut row) in weighted_affine.outer_iter_mut().enumerate() {
509        row.mapv_inplace(|v| v * masses[i]);
510    }
511    let affine_gram = affine.t().dot(&weighted_affine);
512    let affine_gram_pinv = symmetric_pseudoinverse(&affine_gram, "affine function-space Gram")?;
513    let form = weighted_affine
514        .dot(&affine_gram_pinv)
515        .dot(&weighted_affine.t());
516    Ok((&form + &form.t()) * 0.5)
517}
518
519/// Pull a center-value quadratic form back through an evaluation map.
520fn pullback_center_form(evaluation: &Array2<f64>, form: &Array2<f64>) -> Array2<f64> {
521    let pulled = evaluation.t().dot(form).dot(evaluation);
522    (&pulled + &pulled.t()) * 0.5
523}
524
525/// Energy factor `F` of a symmetric PSD center-value form: `FᵀF = H` on `H`'s
526/// positive part, rows `√λ_k · u_kᵀ`.
527///
528/// Every eigenvalue that is positive is kept — no rank tolerance enters, so
529/// this is an exact factorization of `H⁺` rather than a truncation. `H` here is
530/// always the output of [`measure_jet_energy_form`] or the affine projector,
531/// both of which are PSD by construction, so the dropped part is roundoff.
532fn psd_energy_factor(form: &Array2<f64>, context: &str) -> Result<Array2<f64>, BasisError> {
533    let m = form.nrows();
534    if m == 0 {
535        return Ok(Array2::<f64>::zeros((0, 0)));
536    }
537    let sym = (form + &form.t()) * 0.5;
538    let (evals, evecs) = sym.eigh(Side::Lower).map_err(|e| {
539        BasisError::InvalidInput(format!(
540            "measure-jet energy factorization `{context}` eigendecomposition failed: {e}"
541        ))
542    })?;
543    let kept: Vec<usize> = evals
544        .iter()
545        .enumerate()
546        .filter_map(|(index, &value)| (value > 0.0).then_some(index))
547        .collect();
548    let mut factor = Array2::<f64>::zeros((kept.len(), m));
549    for (row, index) in kept.into_iter().enumerate() {
550        let scale = evals[index].sqrt();
551        for column in 0..m {
552            factor[[row, column]] = scale * evecs[[column, index]];
553        }
554    }
555    Ok(factor)
556}
557
558/// Pull a PSD center-value form back through an evaluation map CONSTRUCTIVELY:
559/// `EᵀHE = (F E)ᵀ (F E)` for `FᵀF = H`.
560///
561/// The dense route (`pullback_center_form` + `try_from_dense_psd`) is a false
562/// refusal waiting to happen and #2761 measured it firing: the Gaussian
563/// representers go collinear as the range ℓ grows, `E`'s condition number blows
564/// up, and the triple product loses exactly the digits that keep the smallest
565/// eigenvalue non-negative. On `measure_jet_perf_parity` the whole design then
566/// refuses to build for every `ℓ ≳ 2.8` —
567///
568/// ```text
569///   ell   2.15059  builds        reml -1279.00091366   (still descending)
570///   ell   2.79577  REFUSED  min eigenvalue -9.266e-9
571///   ell   7.98500  REFUSED  min eigenvalue -2.864e-5
572/// ```
573///
574/// — while the criterion is still descending at the last `ℓ` that builds, which
575/// is what the outer search reports as `StepSizeTooSmall after 50 attempt(s)`:
576/// its descent direction points into a region where the objective cannot be
577/// EVALUATED. `-4e-5` relative on a Frobenius-normalized matrix is cancellation,
578/// not negative curvature, and the refusal's own guidance says so ("supply the
579/// native energy factor for a PSD function penalty").
580///
581/// Going through the factor makes PSD-ness structural instead of a numerical
582/// accident, and never squares `E`'s condition number in the middle product.
583/// The module's null-component penalty already worked this way
584/// ([`affine_function_nullspace_quadratic`]); the energy Primary was the
585/// sibling that did not.
586fn constructive_pullback_center_form(
587    evaluation: &Array2<f64>,
588    form: &Array2<f64>,
589    context: &str,
590) -> Result<ConstructiveQuadratic, BasisError> {
591    let factor = psd_energy_factor(form, context)?;
592    ConstructiveQuadratic::from_energy_factor(factor.dot(evaluation), context)
593}
594
595/// Congruence of a jet onto a frame: `F (Fᵀ J F) Fᵀ`.
596///
597/// This is the exact derivative of `R(ψ) = N M(ψ) Nᵀ`, `M = Nᵀ S(ψ) N`, for a
598/// ψ-FIXED frame `N` — which is what a declared structural null frame is. An
599/// empty frame gives an exact zero, matching a rebuild that declined.
600fn restrict_jet_to_frame(jet: &Array2<f64>, frame: &Array2<f64>) -> Array2<f64> {
601    if frame.ncols() == 0 {
602        return Array2::<f64>::zeros(jet.dim());
603    }
604    let inner = frame.t().dot(jet).dot(frame);
605    let restricted = frame.dot(&inner).dot(&frame.t());
606    (&restricted + &restricted.t()) * 0.5
607}
608
609/// Frobenius scale of a constructive quadratic, with the same degenerate
610/// convention `normalize_penalty` uses: a scale at or below `1e-12` reports
611/// `1e-12` so the division can never blow up.
612fn constructive_frobenius_scale(quadratic: &ConstructiveQuadratic) -> f64 {
613    quadratic
614        .dense()
615        .iter()
616        .map(|value| value * value)
617        .sum::<f64>()
618        .sqrt()
619        .max(1e-12)
620}
621
622/// First and diagonal-second `u = ln ℓ` derivatives of `E(u)ᵀ H E(u)` for a
623/// `u`-invariant center-value form `H`.
624fn pullback_center_form_log_length_jets(
625    evaluation: &Array2<f64>,
626    evaluation_first: &Array2<f64>,
627    evaluation_second: &Array2<f64>,
628    form: &Array2<f64>,
629) -> (Array2<f64>, Array2<f64>) {
630    let h_e = form.dot(evaluation);
631    let h_e_first = form.dot(evaluation_first);
632    let h_e_second = form.dot(evaluation_second);
633    let first_raw = evaluation_first.t().dot(&h_e) + evaluation.t().dot(&h_e_first);
634    let second_raw = evaluation_second.t().dot(&h_e)
635        + evaluation.t().dot(&h_e_second)
636        + evaluation_first.t().dot(&h_e_first) * 2.0;
637    (
638        (&first_raw + &first_raw.t()) * 0.5,
639        (&second_raw + &second_raw.t()) * 0.5,
640    )
641}
642
643/// Mixed derivative `∂²(EᵀH(ψ)E)/(∂lnℓ ∂ψ)` when only `E` depends on `ℓ`.
644fn pullback_center_form_log_length_cross(
645    evaluation: &Array2<f64>,
646    evaluation_first: &Array2<f64>,
647    form_first: &Array2<f64>,
648) -> Array2<f64> {
649    let h_e = form_first.dot(evaluation);
650    let h_e_first = form_first.dot(evaluation_first);
651    let cross_raw = evaluation_first.t().dot(&h_e) + evaluation.t().dot(&h_e_first);
652    (&cross_raw + &cross_raw.t()) * 0.5
653}
654
655/// The Primary energy's structural null frame in whatever coefficient chart
656/// `z` realizes: the coefficients whose REPRESENTER block vanishes, i.e. the
657/// pure ambient-affine-head directions.
658///
659/// This is a theorem of the construction, not a measurement, and it is the
660/// reason the double-penalty topology can be ψ-invariant (#2445's mechanism,
661/// applied here for #2761):
662///
663/// * the energy annihilates ambient-affine center values EXACTLY (the module's
664///   no-mass / exact-affine-projection contract), so a coefficient whose center
665///   values are `head_cc·b_head` — pure affine — is annihilated for EVERY `ℓ`;
666/// * the single-scale gauge restricts the representer block to `null(AᵀW K_cc)`,
667///   so its center values are mass-orthogonal to the affine space. A nonzero
668///   representer part therefore cannot land in the energy's null space:
669///   `K_cc z_rbf b_rbf ∈ A ∩ A^⊥ = {0}` forces `z_rbf b_rbf = 0`.
670///
671/// So `null(Primary) = { b : (z·b)|representer rows = 0 }` exactly, in the
672/// fit-time chart AND in any composed frozen chart, because both statements are
673/// about `z` alone. Nothing here reads `ℓ`.
674///
675/// Without the declaration the topology is decided by a rank test on the
676/// pullback `kzᵀ Q kz`, whose numerical rank falls as the range grows and the
677/// representers go collinear (measured on a 1-D 50-center term: rank 47 of 48 at
678/// the auto seed, 14 of 48 at 8x it). A design-moving `ℓ` then adds or removes
679/// the double-penalty ridge between outer trials — the `incremental realizer
680/// topology changed` abort, and the `#860` penalty-count desync class.
681///
682/// Returns `None` in multiscale mode, where there is no affine head: the
683/// energy's null space is then a genuinely `ℓ`-dependent subspace of the
684/// representer block, and declaring an EMPTY frame would assert the opposite of
685/// the truth (that the Primary has no null space at all) and silently delete the
686/// null component. Measuring is the honest fallback there.
687fn measure_jet_primary_structural_null_frame(
688    z: &Array2<f64>,
689    representer_count: usize,
690    head_rank: usize,
691) -> Result<Option<Array2<f64>>, BasisError> {
692    if head_rank == 0 || representer_count == 0 || z.ncols() == 0 {
693        return Ok(None);
694    }
695    // `rrqr_nullspace_basis(B)` returns `null(Bᵀ)`, so pass the transpose of the
696    // representer rows to get the coefficient-space null vectors.
697    let representer_rows = z.slice(ndarray::s![..representer_count, ..]).to_owned();
698    let (frame, _) =
699        rrqr_nullspace_basis(&representer_rows.t().to_owned(), default_rrqr_rank_alpha())
700            .map_err(BasisError::LinalgError)?;
701    // An EMPTY frame is a declaration, not a missing one: it says the chart has
702    // absorbed every affine-head direction, so the Primary has no null space
703    // here. That is the answer in a composed frozen chart, where the global
704    // parametric orthogonalization has already taken the head — and it is
705    // exactly the chart every ψ trial rebuilds in. Returning `None` there would
706    // send the topology decision back to the rank test the declaration exists
707    // to replace, reinstating the ℓ-dependence on the one path that cannot
708    // tolerate it. `None` is reserved for "this construction does not apply"
709    // (multiscale, no head).
710    Ok(Some(frame))
711}
712
713/// Fixed-rank constructive witness for the affine/null quadratic in center-value
714/// space. Rank is decided here, where `H₀` is independent of `ℓ`, rather than
715/// after its coefficient pullback has acquired `ℓ`-dependent roundoff modes.
716fn affine_function_nullspace_center_quadratic(
717    centers: ArrayView2<'_, f64>,
718    masses: ArrayView1<'_, f64>,
719) -> Result<ConstructiveQuadratic, BasisError> {
720    ConstructiveQuadratic::try_from_dense_psd(
721        affine_function_nullspace_form(centers, masses)?,
722        "measure-jet affine center-value form",
723    )
724}
725
726fn affine_function_nullspace_quadratic(
727    evaluation: &Array2<f64>,
728    centers: ArrayView2<'_, f64>,
729    masses: ArrayView1<'_, f64>,
730) -> Result<ConstructiveQuadratic, BasisError> {
731    if evaluation.nrows() != centers.nrows() {
732        crate::bail_dim_basis!(
733            "measure-jet affine function-space penalty shape mismatch: evaluation {:?}, centers {:?}",
734            evaluation.dim(),
735            centers.dim()
736        );
737    }
738    let center_quadratic = affine_function_nullspace_center_quadratic(centers, masses)?;
739    ConstructiveQuadratic::from_energy_factor(
740        center_quadratic.factor().dot(evaluation),
741        "measure-jet affine/null coefficient penalty",
742    )
743}
744
745/// Pairwise squared distances `‖a_i − b_j‖²` via the GEMM identity
746/// `‖a − b‖² = ‖a‖² + ‖b‖² − 2·aᵀb`: one (n×d)·(d×m) matrix product carries
747/// every FMA at tile speed instead of n·m scalar distance loops — the
748/// machine-native form of this kernel, and the module's ONLY distance
749/// source (representer design, support curve, and the center-pair geometry:
750/// band floor, median spacing, ε/2-net, neighbor cutoffs). The cancellation
751/// error near-coincident points pay is O(ε_f64·‖x‖²) ABSOLUTE, harmless
752/// under a Gaussian profile (the kernel is flat at d ≈ 0); clamped at zero
753/// so roundoff cannot emit tiny negatives (the a = b diagonal therefore
754/// lands at roundoff scale, not an exact 0 — no caller pins it).
755pub(crate) fn pairwise_sq_dists(a: ArrayView2<'_, f64>, b: ArrayView2<'_, f64>) -> Array2<f64> {
756    let an: Vec<f64> = a.outer_iter().map(|r| r.dot(&r)).collect();
757    let bn: Vec<f64> = b.outer_iter().map(|r| r.dot(&r)).collect();
758    let mut g = a.dot(&b.t());
759    g.axis_iter_mut(Axis(0))
760        .into_par_iter()
761        .enumerate()
762        .for_each(|(i, mut row)| {
763            for (j, v) in row.iter_mut().enumerate() {
764                *v = (an[i] + bn[j] - 2.0 * *v).max(0.0);
765            }
766        });
767    g
768}
769
770/// Row-block size for streaming GEMM passes that must not materialize the
771/// full n×m distance matrix (nearest-node assignment): 64Ki rows × m ≤ a
772/// few hundred MB of transient per block, GEMM-speed throughout.
773pub(crate) const MEASURE_JET_ASSIGN_BLOCK_ROWS: usize = 65_536;
774
775pub(crate) fn validate_finite_points(
776    points: ArrayView2<'_, f64>,
777    what: &str,
778) -> Result<(), BasisError> {
779    for (i, row) in points.outer_iter().enumerate() {
780        if row.iter().any(|v| !v.is_finite()) {
781            crate::bail_invalid_basis!("measure-jet {what} row {i} has a non-finite coordinate");
782        }
783    }
784    Ok(())
785}
786
787/// Median nearest-OTHER-center distance — the resolution floor of the center
788/// quadrature, used for the band floor and the auto representer range.
789pub(crate) fn median_nearest_center_spacing(dist2: &Array2<f64>) -> Result<f64, BasisError> {
790    let m = dist2.nrows();
791    if m < 2 {
792        return Err(BasisError::InsufficientColumnsForConstraint { found: m });
793    }
794    let mut nearest: Vec<f64> = Vec::with_capacity(m);
795    for i in 0..m {
796        let mut best = f64::INFINITY;
797        for j in 0..m {
798            if j != i && dist2[(i, j)] < best {
799                best = dist2[(i, j)];
800            }
801        }
802        nearest.push(best.sqrt());
803    }
804    nearest.sort_by(|a, b| a.partial_cmp(b).expect("finite center spacings"));
805    let median = nearest[nearest.len() / 2];
806    if !(median.is_finite() && median > 0.0) {
807        crate::bail_invalid_basis!(
808            "measure-jet centers are degenerate (median nearest-center spacing = {median}); \
809             duplicate centers cannot carry a scale band"
810        );
811    }
812    Ok(median)
813}
814
815/// Make the representer coefficient section a BASIS of its span at the realized
816/// range, not merely a spanning set (gam#2750).
817///
818/// ## What goes wrong without this
819///
820/// A Gaussian kernel's spectrum decays super-exponentially in `ℓ/spacing`, so
821/// the center evaluation map `E = K_cc · Z_rbf` — through which BOTH the design
822/// and every penalty pullback pass — loses conditioning fast as the representer
823/// range grows. The raw section is chosen for head-orthogonality alone and
824/// keeps every direction, so the shipped chart carries directions the criterion
825/// cannot resolve, and the criterion's own Occam term is the first casualty:
826///
827/// ```text
828///   ell     cond(S)     d log|S|+/d ln ell:  analytic       FD        gap
829///   1.21    1.2e10                          -109.843    -109.829    -0.014
830///   1.57    3.0e11                          -107.924    -107.919    -0.005
831///   2.04    3.7e13                           -97.252     -96.331    -0.922
832///   2.65    3.8e13                           -51.836     -64.765   +12.930
833///   3.45    6.5e12  (rank 14 -> 13)          -26.621     -40.633   +14.012
834/// ```
835///
836/// (16 centers on the `measure_jet_perf_parity` geometry, chart frozen so the
837/// finite difference and the analytic jet differentiate the same object; the
838/// jet itself agrees with a central difference of the shipped penalty to
839/// `1e-8` throughout, so the producer is exact and it is `log|S|₊` that stops
840/// being a function.) Once `cond(S)` reaches the rank-classification floor the
841/// kept spectrum bottoms out in roundoff: `log|S|₊` is then a sum over
842/// directions whose logs are noise, `tr(S⁺Ṡ)` divides by them, and the outer
843/// search is handed a direction that is not a descent direction of its own
844/// objective. That is the wall the `ln ℓ` coordinate has been hitting.
845///
846/// ## The section
847///
848/// Whiten against the section's own center evaluation map,
849/// `E = K_cc Z = U Σ Vᵀ`: take `Z ← Z · V · diag(1/max(σ, √ε·‖K_cc‖))`, so the
850/// realized section satisfies `EᵀE = I` wherever `E` is resolvable and is
851/// *damped rather than deleted* below that. Two questions, two bars, and they
852/// are not the same question — but they share one anchor, and the anchor is
853/// `‖K_cc‖₂`, the norm of the operator whose product formed `E`, never `E`'s
854/// own largest singular value (which collapses; see the table at the bars):
855///
856/// * **Does the direction exist?** `σ > ε·‖K_cc‖·max(dim)` — the backward-error
857///   bar of forming `E = K_cc·Z`. Below it there is no direction, only the
858///   roundoff of that product.
859/// * **How far may it be amplified?** `1/σ` is also the factor by which
860///   direction `i` amplifies the design's own roundoff, and the criterion
861///   squares the design into `XᵀWX`, so the amplification survives that Gram
862///   with significant digits exactly when `ε·(‖K_cc‖/σ_i)² < 1`. Hence the
863///   scaling — not the membership — is floored at `√ε·‖K_cc‖`.
864///
865/// **Damping instead of deleting is what makes the width `ℓ`-invariant.** A
866/// span is invariant under ANY invertible diagonal rescaling, so a damped
867/// direction contributes exactly the same column space as an undamped one; it
868/// simply enters the design with a small norm instead of an amplified one, and
869/// carries its own roundoff in at that same small norm. The realized chart
870/// therefore keeps every direction `E` has, at every `ℓ`, while
871/// `cond(E·W) ≤ √ε·cond(E)` is bounded by construction.
872///
873/// * The rescaling is a pure change of coefficient chart. The profiled
874///   criterion is invariant under an invertible reparameterization (`X → XT`,
875///   `S → TᵀST` moves `log|XᵀWX + λS|` and `log|λS|₊` by the same
876///   `2 ln|det T|`), so it changes no estimate — only the arithmetic.
877/// * **The whitening removes the squaring from the PENALTY, exactly.** With
878///   `EᵀE = I` the energy pullback is `S = UᵀQU` with `U` orthonormal, whose
879///   spectrum is bounded by `Q`'s; and `Q` is `ℓ`-INVARIANT (a form on center
880///   VALUES), so `cond(S)` stops being a function of the range at all.
881///   Measured, re-realizing the chart at each `ℓ` on the #2761 fixture:
882///   `cond(E) = 1.0`, `cond(S₊) = 21.1`, `log|S|₊ = 0.800` from `1×` to `16×`
883///   the seed range. So "the energy pullback squares `E`" — the reason the
884///   previous cut gave for being where it was — does not survive its own
885///   remedy.
886/// * **What the whitening does NOT remove is the squaring in `XᵀWX`.** The
887///   chart lives at the CENTERS; the design lives at the DATA, and `1/σ_i` is
888///   also the factor by which direction `i` amplifies the design's own
889///   roundoff. The criterion squares the design, so a retained direction
890///   survives with significant digits exactly when `ε·(σ_max/σ_i)² < 1`. That
891///   is where the half-mantissa belongs, and it is one half-mantissa, not two.
892/// * **The previous cut spent the half-mantissa twice, on an already-squared
893///   quantity — and spent it by DELETING.** It whitened against `G = EᵀE` and
894///   cut at `√ε·λ_max(G)`; since `λ = σ²` that is `σ > ε^{1/4}·σ_max`, i.e.
895///   `cond(E) ≤ ε^{-1/4} ≈ 8·10³`. Measured on the #2761 fixture at 16 centers,
896///   that deleted most of the span at the ranges REML actively selects:
897///
898/// ```text
899///   ℓ/ℓ_seed  cond(E)   ε^{1/4} DELETE: p  span floor    √ε DAMP: p  span floor
900///      1      3.0e+01         12            6.11e-2          12       6.11e-2
901///      2      2.8e+04         11            2.43e-2          12       1.94e-2
902///      4      4.2e+07          8            1.50e-2          12       2.10e-3
903///      8      9.1e+09          6            1.67e-2          12       1.81e-4   <- 92x
904///     16      2.7e+11          4            8.92e-2          12       3.54e-5
905/// ```
906///
907///   `span floor` is the least-squares residual RMSE of the NOISELESS truth on
908///   the realized design's own column span — the bound no `λ` can beat, since
909///   `λ` shrinks inside a span and never moves one. Note the `ε^{1/4}` floor
910///   going UP past `4×`: that chart is worse than the seed range it was meant
911///   to improve on. The damped column keeps the whole span (`1.81e-4` at `8×`
912///   reproduces an 80-digit projection of the same span to every digit) at a
913///   width that does not move with the dial, which a DELETING bar cannot do at
914///   any threshold — `12,11,8,6,4` for `ε^{1/4}` and `12,12,12,11,10` even for
915///   `√ε`.
916///
917/// Realized per cold build from `K_cc(ℓ)`, so the chart tracks the dial it is a
918/// chart for; a frozen-quadrature replay reuses the composed transform verbatim.
919fn condition_representer_section(
920    k_cc: &Array2<f64>,
921    z_rbf: &Array2<f64>,
922) -> Result<Array2<f64>, BasisError> {
923    if z_rbf.ncols() == 0 {
924        return Ok(z_rbf.clone());
925    }
926    let evaluation = k_cc.dot(z_rbf);
927    let (_, singular, right) = evaluation.svd(false, true).map_err(BasisError::LinalgError)?;
928    let leading = singular.iter().copied().fold(0.0_f64, f64::max);
929    if !(leading.is_finite() && leading > 0.0) {
930        return Ok(z_rbf.clone());
931    }
932    // `right` is `Vᵀ`: row `i` is `σ_i`'s right singular vector, in the
933    // coefficient coordinates of `z_rbf`.
934    let right = right.ok_or_else(|| {
935        BasisError::LinalgError(gam_linalg::faer_ndarray::FaerLinalgError::SvdNoConvergence {
936            context: "measure-jet representer section: right singular vectors were not returned",
937        })
938    })?;
939    // Both bars are anchored on `‖K_cc‖₂`, NOT on `σ_max(E)`. `E = K_cc·Z` with
940    // orthonormal `Z`, so the computed `E` carries a backward error
941    // `O(ε·‖K_cc‖₂)` whatever `E`'s own size turns out to be — and `σ_max(E)`
942    // is not a fixed fraction of `‖K_cc‖₂`: it COLLAPSES as `ℓ` grows, because
943    // the constraint that makes `Z` head-orthogonal is exactly what annihilates
944    // the flat limit `K_cc → 𝟙𝟙ᵀ`. Measured on the 1-D sweep fixture (50
945    // centers, standardized `x`):
946    //
947    // ```text
948    //   ℓ      ‖K_cc‖₂  σ_max(E)  σ_max/‖K‖  σ_min(E)
949    //   0.05     2.4     2.29e+0   9.6e-1     5.5e-2
950    //   0.55    17.2     8.79e+0   5.1e-1     3.2e-16   <- σ_min is the roundoff
951    //   6.09    48.4     1.49e-2   3.1e-4     4.5e-17      floor, and it is FLAT
952    //  49.73    50.0     3.48e-6   7.0e-8     3.5e-17
953    // ```
954    //
955    // A bar relative to `σ_max(E)` therefore falls BELOW that flat roundoff
956    // floor once the range is long, and admits the floor itself as signal: at
957    // `ℓ = 11` a `ε·σ_max·dim` bar is `1.5e-17` against entries of `1.7e-17`,
958    // so all 48 directions "pass" and the chart hands the fit 48 columns of
959    // pure rounding noise. Those columns fit anything, so the criterion has a
960    // spurious minimum out there — measured at `V = −447.8` on this fixture's
961    // sweep case 1 against `≈ −44` in the honest region, which is precisely
962    // where its outer search was terminating.
963    let dimension_factor = evaluation.nrows().max(evaluation.ncols()) as f64;
964    let (_, kernel_singular, _) = k_cc.svd(false, false).map_err(BasisError::LinalgError)?;
965    let anchor = kernel_singular
966        .iter()
967        .copied()
968        .fold(0.0_f64, f64::max)
969        .max(leading);
970    // Membership: the backward-error bar of the product that formed `E`. Below
971    // it there is no direction, only the roundoff of `K_cc·Z`.
972    let existence = anchor * f64::EPSILON * dimension_factor;
973    // Amplification: ONE half-mantissa, spent ONCE. `1/σ_i` is the factor by
974    // which direction `i` lifts the design's own roundoff — which is set by
975    // `‖K_xc‖ ∼ ‖K_cc‖`, not by `σ_max(E)` — and the criterion squares the
976    // design into `XᵀWX`, so the lift survives that Gram with significant
977    // digits exactly when `ε·(‖K_cc‖/σ_i)² < 1`. Below `√ε·‖K_cc‖` the
978    // direction is DAMPED to that scaling rather than dropped: the span is
979    // unchanged (any invertible rescaling spans the same columns) and the
980    // roundoff comes in at the damped norm instead of an amplified one.
981    //
982    // This also reproduces the range bracket's own physics for free. As `ℓ`
983    // passes the node diameter, `σ_max(E)` falls under the floor and the WHOLE
984    // representer block damps smoothly toward zero, leaving the affine head —
985    // which is exactly `MeasureJetRangeBracket::ceiling`'s statement that past
986    // there "the block is numerically one function plus the affine head and
987    // there is no distinct model past it", now realized by the arithmetic
988    // rather than asserted next to it.
989    let amplification_floor = anchor * f64::EPSILON.sqrt();
990    // Visibility: a damped direction enters the energy pullback
991    // `S = (E·W)ᵀQ(E·W)` with squared weight `(σ_i/floor)²`, so if that weight
992    // drops below the canonical penalty-spectrum rank cutoff the direction is
993    // classified UNPENALIZED — an accidentally free design direction, which is
994    // the opposite of conservative. It also makes `log|S|₊` a step function of
995    // `ℓ`: measured on the 1-D sweep fixture, the primary's nullity flapping by
996    // one moved the profiled criterion by `8.5` at fixed `λ`, which is what a
997    // `ln ℓ` line search cannot cross.
998    //
999    // So the chart's retention bar is at least as strict as the penalty's own:
1000    // keep direction `i` only while `(σ_i/floor)² > tol`, with `tol` the same
1001    // `spectral_tolerance` convention (#1425's single classifier) every other
1002    // penalty-spectrum consumer reads. No second constant, and the two
1003    // decisions can no longer disagree about which directions are penalized.
1004    let rank_tolerance =
1005        z_rbf.ncols().max(1) as f64 * super::bspline_build::SPECTRAL_RANK_RELATIVE_TOLERANCE;
1006    let visibility = amplification_floor * rank_tolerance.sqrt();
1007    let retention = existence.max(visibility);
1008    let kept: Vec<usize> = (0..singular.len())
1009        .filter(|&i| singular[i] > retention)
1010        .collect();
1011    let kept = if kept.is_empty() {
1012        // Every representer direction is below the resolvable floor. Keep the
1013        // single strongest one rather than emitting an empty block: a term with
1014        // no representer columns is a different model, and that decision
1015        // belongs to the range screen, not to a conditioning step.
1016        vec![(0..singular.len()).fold(0usize, |best, i| {
1017            if singular[i] > singular[best] { i } else { best }
1018        })]
1019    } else {
1020        kept
1021    };
1022    let mut transform = Array2::<f64>::zeros((z_rbf.ncols(), kept.len()));
1023    for (column, &index) in kept.iter().enumerate() {
1024        let inverse = singular[index]
1025            .max(amplification_floor)
1026            .max(f64::MIN_POSITIVE)
1027            .recip();
1028        // Sign gauge: a singular vector is defined up to sign, and the sign a
1029        // decomposition happens to return is not a property of the geometry.
1030        // Pin it on the entry of largest magnitude so the realized chart is
1031        // reproducible across faer revisions and platforms.
1032        let mut pivot = 0usize;
1033        for row in 1..z_rbf.ncols() {
1034            if right[(index, row)].abs() > right[(index, pivot)].abs() {
1035                pivot = row;
1036            }
1037        }
1038        let sign = if right[(index, pivot)] < 0.0 { -1.0 } else { 1.0 };
1039        for row in 0..z_rbf.ncols() {
1040            transform[(row, column)] = sign * right[(index, row)] * inverse;
1041        }
1042    }
1043    Ok(z_rbf.dot(&transform))
1044}
1045
1046/// Axis-aligned bounding-box diagonal of a point set — the deterministic
1047/// diameter proxy the scale band and the range bracket both measure the
1048/// configuration's extent with. `O(m·d)` and permutation-invariant, unlike a
1049/// max-pairwise-distance scan.
1050pub(crate) fn bounding_box_diagonal(points: ArrayView2<'_, f64>) -> f64 {
1051    let mut diag2 = 0.0_f64;
1052    for k in 0..points.ncols() {
1053        let col = points.column(k);
1054        let mut lo = f64::INFINITY;
1055        let mut hi = f64::NEG_INFINITY;
1056        for &v in col.iter() {
1057            lo = lo.min(v);
1058            hi = hi.max(v);
1059        }
1060        if lo.is_finite() && hi.is_finite() {
1061            diag2 += (hi - lo) * (hi - lo);
1062        }
1063    }
1064    diag2.sqrt()
1065}
1066
1067/// The deterministic bracket the representer range `ℓ` is SCREENED over before
1068/// the outer ψ search refines it, in the STANDARDIZED frame the basis is
1069/// realized in (gam#2750).
1070///
1071/// ## Why a bracket exists at all
1072///
1073/// `ℓ` is a design-moving coordinate: it decides WHICH span the representers
1074/// occupy, and the outer search reaches it by local descent from the seed. The
1075/// profiled criterion in `ln ℓ` is not unimodal — as `ℓ` grows past a few
1076/// center spacings the Gaussian columns become collinear, the rank-revealing
1077/// identifiability section drops columns, and the criterion steps. Measured on
1078/// `measure_jet_formula_fit_robustness_sweep` seed 1 (n = 200, one sine cycle):
1079/// a local minimum at the auto range `ℓ = 0.020` (`V = −234.5`), a barrier at
1080/// `ℓ = 0.035` (`V = −231.1`), and the GLOBAL minimum at `ℓ = 0.80`
1081/// (`V = −256.3`) — 21.7 log units deeper, with held-out RMSE `0.0084` against
1082/// `0.0175`, i.e. the same basis fitting 2.1× better and beating `tp` on both
1083/// the criterion and the truth instead of losing on both. A local descent
1084/// seeded inside the first basin cannot cross that barrier, and the frozen
1085/// coefficient chart the ψ trials rebuild in stops being evaluable ~1.6× past
1086/// the seed, so the search terminates essentially where it started. The λ that
1087/// comes back is then a faithful readout of a range nothing could move — which
1088/// is the "1-D fits select a too-large λ" this bracket exists to end.
1089///
1090/// ## Why THIS bracket
1091///
1092/// The nodes are the term's own realized scale band, verbatim. That is not a
1093/// coincidence of convenience: the energy's `ε` and the representer range `ℓ`
1094/// are the same physical quantity — a length in the chart — and the band is
1095/// already derived, not chosen: its floor is the median nearest-node spacing
1096/// (below it neighbouring representers stop overlapping and the design is a
1097/// bump-per-node indicator with no partition of unity) and its ceiling is half
1098/// the node bounding-box diagonal, at the band's own auto-clamped resolution.
1099/// So the screen introduces no length, no count and no step of its own.
1100///
1101/// ## Where a walk past the top node stops (#2761)
1102///
1103/// [`MeasureJetRangeBracket::feasibility_ceiling`] — the range at which the
1104/// closest node pair stops being distinguishable in the chart's own arithmetic,
1105/// [`measure_jet_range_feasibility_ceiling`]. It is the SAME wall
1106/// [`measure_jet_ln_range_window`] gives the outer search, and it is the same
1107/// wall for the same reason: a stopping rule may not be tighter than the model.
1108///
1109/// It used to be [`MeasureJetRangeBracket::node_diameter`], on the argument
1110/// that at `ℓ` that long every pair of representers overlaps at `≥ exp(−1/2)`
1111/// so "there is no distinct model past it". That argument is measurably wrong,
1112/// and the tree said so in two places before this: `measure_jet_ln_range_window`
1113/// records that *"the profiled criterion genuinely prefers a range AT or ABOVE
1114/// the node diameter"* on three fixtures, and
1115/// `the_search_window_reaches_past_where_the_screen_stops_walking` pins the
1116/// search window as strictly wider. Those reconcile only while something else
1117/// keeps searching past the stopping rule. On a term whose `ℓ` dial is FROZEN —
1118/// the BMS marginal/log-slope pair, or any `learn_length_scale=false` — nothing
1119/// does, and the stopping rule becomes the wall.
1120///
1121/// Measured on the #1041 parity fixture (`m = 10`, extent `[2.671, 2.726]`):
1122/// band `[1.08074, 1.43607, 1.90823]`, `log_step = 0.284265`, node diameter
1123/// `3.81645`. The screen's chosen range for the marginal surface was `3.36930`
1124/// — walk node 2 to every printed digit, with walk node 3 at `4.47708` past the
1125/// diameter. The walk pushes a node and only then breaks if it failed to
1126/// improve, so an argmin that IS the last pushed node improved: the walk left
1127/// through the ceiling test with the criterion still descending.
1128///
1129/// **What raising the stop was actually worth here, measured after the change,
1130/// because it is less than the shape of the defect suggests.** The walk now
1131/// scores `4.47708`, which does NOT improve — so the criterion has an interior
1132/// optimum on this fixture and the old ceiling happened to cut just past it.
1133/// What the extra node buys is the PARABOLIC REFINEMENT, which cannot fire on
1134/// an argmin that is the last element: with a neighbour on both sides the
1135/// refinement lands at `ℓ = 3.10543` with a better criterion value, and
1136/// held-out marginal RMSE goes `0.04185 → 0.04179`. A rule that cannot be
1137/// stepped past also cannot be refined at, and that is the part of the cost
1138/// that was invisible.
1139///
1140/// The larger held-out number on the same sweep — `0.03788` at `ℓ = 68.5`,
1141/// where the block still carries `edf = 7.47` and is not degenerate — is NOT
1142/// what this change recovers, and attributing it to the ceiling would be wrong:
1143/// the criterion does not want to go there. That gap is a statement about the
1144/// screening CRITERION (a profiled Gaussian REML of the term alone against a
1145/// binary response) disagreeing with held-out truth at long ranges, and it
1146/// belongs to whoever takes that question next.
1147///
1148/// Raising the stop is safe by the walk's own rule, which only continues while
1149/// the criterion improves: on the gam#2750 fixture, where the criterion drops
1150/// from `−256.3` to `−198.5` just past the diameter, the walk stops on the
1151/// first non-improving node exactly as it does today. The ceiling only ever
1152/// binds where the criterion is still descending — which is precisely the case
1153/// where stopping is wrong.
1154#[derive(Clone, Debug)]
1155pub struct MeasureJetRangeBracket {
1156    /// Geometric grid of candidate ranges, ascending. The realized scale band.
1157    pub nodes: Vec<f64>,
1158    /// The band's own log step, so an endpoint walk keeps its resolution.
1159    pub log_step: f64,
1160    /// The node bounding-box diagonal. A geometric fact about the cloud,
1161    /// reported because the band's own ceiling is half of it; NOT a stopping
1162    /// rule (see the type docs).
1163    pub node_diameter: f64,
1164    /// Hard upper end for any walk past the top node: the feasibility wall
1165    /// [`measure_jet_range_feasibility_ceiling`], the same one the outer
1166    /// search's [`measure_jet_ln_range_window`] stops at.
1167    pub feasibility_ceiling: f64,
1168}
1169
1170/// The range at which a node pair separated by `spacing` stops being
1171/// DISTINGUISHABLE from a coincident one in `f64`.
1172///
1173/// `exp(−spacing²/2ℓ²)` has come within `√ε` of 1 at
1174/// `ℓ = spacing/√(2√ε)`; past it `K_cc` is the all-ones matrix to working
1175/// precision and the gauge annihilates exactly that (the affine span, constant
1176/// included), so no distinct model survives. `√ε` is the chart's own bar — the
1177/// same half-mantissa `condition_representer_section` spends, and for the same
1178/// reason: it is the point past which a direction cannot survive being squared
1179/// into a Gram and inverted back out.
1180///
1181/// ONE definition, used by both the outer search's window
1182/// ([`measure_jet_ln_range_window`]) and the response screen's walk stop
1183/// ([`MeasureJetRangeBracket::feasibility_ceiling`]), so the two cannot drift
1184/// into disagreeing about where the model ends (#2761).
1185pub fn measure_jet_range_feasibility_ceiling(spacing: f64) -> f64 {
1186    spacing / (2.0 * f64::EPSILON.sqrt()).sqrt()
1187}
1188
1189/// Realize [`MeasureJetRangeBracket`] for a fresh (unfrozen) measure-jet spec.
1190///
1191/// `data` must already be in the standardized frame the basis is built in, and
1192/// the spec must be the FRESH one (no frozen quadrature): the bracket is a
1193/// seeding device and a frozen term has nothing left to seed.
1194pub fn measure_jet_range_bracket(
1195    data: ArrayView2<'_, f64>,
1196    spec: &MeasureJetBasisSpec,
1197) -> Result<MeasureJetRangeBracket, BasisError> {
1198    if spec.frozen_quadrature.is_some() {
1199        crate::bail_invalid_basis!(
1200            "measure-jet range bracket is a seeding device; a frozen-quadrature spec has no seed left to choose"
1201        );
1202    }
1203    if data.ncols() == 0 {
1204        crate::bail_invalid_basis!("measure-jet range bracket needs at least one feature column");
1205    }
1206    validate_finite_points(data, "data")?;
1207    let seed_centers = select_centers_by_strategy(data, &spec.center_strategy)?;
1208    if seed_centers.nrows() < 3 {
1209        return Err(BasisError::InsufficientColumnsForConstraint {
1210            found: seed_centers.nrows(),
1211        });
1212    }
1213    let (nodes, _masses) = measure_jet_quadrature_nodes(data, seed_centers.view())?;
1214    let band = measure_jet_band(nodes.view(), spec.num_scales)?;
1215    // The band floor IS the median nearest-node spacing, so the feasibility
1216    // wall is read off the bracket's own first node rather than remeasured.
1217    let feasibility_ceiling = measure_jet_range_feasibility_ceiling(band.eps[0]);
1218    Ok(MeasureJetRangeBracket {
1219        nodes: band.eps,
1220        log_step: band.log_step,
1221        node_diameter: bounding_box_diagonal(nodes.view()),
1222        feasibility_ceiling,
1223    })
1224}
1225
1226/// The `ln ℓ` SEARCH WINDOW for the design-moving representer range, in the
1227/// frame `spec.length_scale` is expressed in (gam#2750).
1228///
1229/// ## Why this is not an absolute interval
1230///
1231/// `ℓ` is a LENGTH in the chart the basis is realized in, so a window for it is
1232/// a statement about the node cloud, not about `f64`. Both ends here are the
1233/// same measured length — the median nearest-node spacing `s`, which is also
1234/// [`realized_measure_jet_length_scale`]'s auto value and
1235/// [`MeasureJetBand::eps`]'s floor — read at the two ranges where the kernel
1236/// stops saying anything about the pair it separates:
1237///
1238/// * **floor `ℓ = s`.** Neighbouring representers overlap at exactly
1239///   `exp(−1/2)`. Below it they stop overlapping, the design degenerates from a
1240///   partition of unity into a bump-per-node indicator, and rows between nodes
1241///   fall outside every representer's support.
1242/// * **ceiling `ℓ = s/√(2√ε)`.** The same neighbouring pair's kernel value
1243///   `exp(−s²/2ℓ²)` has come within `√ε` of 1, so the pair is no longer
1244///   DISTINGUISHABLE from a coincident one in the arithmetic the chart is built
1245///   in. Past it `K_cc` is the all-ones matrix to working precision, and the
1246///   gauge annihilates exactly that (the affine span, constant included), so
1247///   there is no distinct model left. `√ε` is the chart's own bar — the same
1248///   half-mantissa `condition_representer_section` spends, and for the same
1249///   reason: it is the point past which a direction cannot survive being
1250///   squared into a Gram and inverted back out.
1251///
1252/// The window is therefore `[ln s, ln s − ½ln(2√ε)]`: it TRANSLATES with the
1253/// chart (both ends are proportional to a measured length, so an isotropic
1254/// rescale by `c` shifts them both by `ln c`) and its WIDTH is `8.664`, a pure
1255/// function of `f64::EPSILON` rather than a number anybody picked.
1256///
1257/// ## What this deliberately is NOT
1258///
1259/// It is **not** [`MeasureJetRangeBracket::node_diameter`], the node bounding-box
1260/// diagonal. That is where the response SCREEN stops walking, which is a
1261/// stopping rule for a search over nodes, not a wall in the model: measured on
1262/// three fixtures (`measure_jet_formula_fit_robustness_sweep` seed 1,
1263/// `measure_jet_web_quality`, and the two probes that score them), the profiled
1264/// criterion genuinely prefers a range AT or ABOVE the node diameter, and a box
1265/// that stopped there railed the outer search and refused the fit. A long range
1266/// is a legitimate model — as `ℓ` grows the gauge-quotiented representer span
1267/// tends to a polynomial one, which is exactly the right basis for a smooth
1268/// target — so the upper end has to be a feasibility statement and nothing
1269/// weaker.
1270pub fn measure_jet_ln_range_window(
1271    data: ArrayView2<'_, f64>,
1272    spec: &MeasureJetBasisSpec,
1273) -> Result<(f64, f64), BasisError> {
1274    let spacing = match (&spec.center_strategy, &spec.frozen_quadrature) {
1275        (CenterStrategy::UserProvided(_), Some(frozen)) if !frozen.eps_band.is_empty() => {
1276            frozen.eps_band[0]
1277        }
1278        _ => measure_jet_range_bracket(data, spec)?.nodes[0],
1279    };
1280    if !(spacing.is_finite() && spacing > 0.0) {
1281        crate::bail_invalid_basis!(
1282            "measure-jet ln-range window is degenerate: the node cloud reports a nearest-node \
1283             spacing of {spacing}, so it has no range scale to search over"
1284        );
1285    }
1286    // The range at which `1 - exp(-s^2/2l^2)` reaches the chart's half-mantissa
1287    // bar, from the single definition the screen's walk stop also reads.
1288    let ceiling = measure_jet_range_feasibility_ceiling(spacing);
1289    Ok((spacing.ln(), ceiling.ln()))
1290}
1291
1292/// Build the realized geometric scale band from the center set: floor at the
1293/// median nearest-center spacing (below it the quadrature resolves nothing),
1294/// ceiling at half the bounding-box diagonal (a deterministic diameter-scale
1295/// cap; local fits remain center-weighted and distinct there).
1296/// `num_scales == 0` requests the auto count `clamp(⌈log2(ε_max/ε_min)⌉ + 1,
1297/// 3, 8)`; a degenerate band (ceiling ≤ floor) collapses to the single floor
1298/// scale with `log_step = ln 2`.
1299pub fn measure_jet_band(
1300    centers: ArrayView2<'_, f64>,
1301    num_scales: usize,
1302) -> Result<MeasureJetBand, BasisError> {
1303    validate_finite_points(centers, "centers")?;
1304    let dist2 = pairwise_sq_dists(centers, centers);
1305    let eps_min = median_nearest_center_spacing(&dist2)?;
1306    // Half the bounding-box diagonal: a cheap, deterministic diameter proxy.
1307    let eps_max = 0.5 * bounding_box_diagonal(centers);
1308    if !(eps_max.is_finite() && eps_max > eps_min) {
1309        return Ok(MeasureJetBand {
1310            eps: vec![eps_min],
1311            log_step: std::f64::consts::LN_2,
1312        });
1313    }
1314    let auto = ((eps_max / eps_min).log2().ceil() as usize + 1)
1315        .clamp(MEASURE_JET_MIN_AUTO_SCALES, MEASURE_JET_MAX_AUTO_SCALES);
1316    let count = if num_scales == 0 { auto } else { num_scales };
1317    if count == 1 {
1318        return Ok(MeasureJetBand {
1319            eps: vec![eps_min],
1320            log_step: std::f64::consts::LN_2,
1321        });
1322    }
1323    let ratio = (eps_max / eps_min).powf(1.0 / (count as f64 - 1.0));
1324    let mut eps = Vec::with_capacity(count);
1325    let mut e = eps_min;
1326    for _ in 0..count {
1327        eps.push(e);
1328        e *= ratio;
1329    }
1330    Ok(MeasureJetBand {
1331        eps,
1332        log_step: ratio.ln(),
1333    })
1334}
1335
1336/// First-moment-exact quadrature of the empirical measure on the cell partition
1337/// induced by the seed centers: nearest-center assignment (deterministic
1338/// tie-break: lowest center index) yields per-cell masses, and each non-empty
1339/// cell's quadrature node is its mass-weighted barycenter. Empty cells keep
1340/// their seed coordinates with zero mass (the assembly skips them; their
1341/// representer columns remain valid).
1342pub fn measure_jet_quadrature_nodes(
1343    data: ArrayView2<'_, f64>,
1344    centers: ArrayView2<'_, f64>,
1345) -> Result<(Array2<f64>, Array1<f64>), BasisError> {
1346    if data.ncols() != centers.ncols() {
1347        crate::bail_dim_basis!(
1348            "measure-jet mass assignment dimension mismatch: data d={} centers d={}",
1349            data.ncols(),
1350            centers.ncols()
1351        );
1352    }
1353    validate_finite_points(data, "data")?;
1354    validate_finite_points(centers, "centers")?;
1355    let n = data.nrows();
1356    let m = centers.nrows();
1357    let d = centers.ncols();
1358    if n == 0 || m == 0 {
1359        crate::bail_invalid_basis!("measure-jet mass assignment needs nonempty data and centers");
1360    }
1361    // Nearest-node assignment in streamed GEMM blocks: argmin_j ‖x−c_j‖² =
1362    // argmin_j (‖c_j‖² − 2·xᵀc_j), so each block is one (rows×d)·(d×m)
1363    // product plus a row-wise argmin — tile-speed FMAs, O(block·m) transient
1364    // memory, deterministic ties to the lowest center index.
1365    let cn: Vec<f64> = centers.outer_iter().map(|r| r.dot(&r)).collect();
1366    let assignments: Vec<usize> = (0..n)
1367        .step_by(MEASURE_JET_ASSIGN_BLOCK_ROWS)
1368        .flat_map(|start| {
1369            let end = (start + MEASURE_JET_ASSIGN_BLOCK_ROWS).min(n);
1370            let g = data.slice(ndarray::s![start..end, ..]).dot(&centers.t());
1371            let block: Vec<usize> = g
1372                .axis_iter(Axis(0))
1373                .into_par_iter()
1374                .map(|row| {
1375                    let mut best_j = 0usize;
1376                    let mut best = f64::INFINITY;
1377                    for (j, &gij) in row.iter().enumerate() {
1378                        let s = cn[j] - 2.0 * gij;
1379                        if s < best {
1380                            best = s;
1381                            best_j = j;
1382                        }
1383                    }
1384                    best_j
1385                })
1386                .collect();
1387            block
1388        })
1389        .collect();
1390    let mut masses = Array1::<f64>::zeros(m);
1391    let mut nodes = centers.to_owned();
1392    let mut sums = Array2::<f64>::zeros((m, d));
1393    let unit = 1.0 / n as f64;
1394    for (i, &j) in assignments.iter().enumerate() {
1395        masses[j] += unit;
1396        for k in 0..d {
1397            sums[(j, k)] += data[(i, k)];
1398        }
1399    }
1400    // Cell barycenters: the first moment of μ on each cell. These are the
1401    // realized nodes for first-moment-exact lumping.
1402    let mut barycenter = sums;
1403    for j in 0..m {
1404        let count = masses[j] * n as f64;
1405        if count > 0.0 {
1406            for k in 0..d {
1407                barycenter[(j, k)] /= count;
1408                nodes[(j, k)] = barycenter[(j, k)];
1409            }
1410        }
1411    }
1412    Ok((nodes, masses))
1413}
1414
1415/// Per-center masses of the empirical measure (the zeroth-moment half of
1416/// [`measure_jet_quadrature_nodes`]; single assignment source).
1417pub fn measure_jet_center_masses(
1418    data: ArrayView2<'_, f64>,
1419    centers: ArrayView2<'_, f64>,
1420) -> Result<Array1<f64>, BasisError> {
1421    measure_jet_quadrature_nodes(data, centers).map(|(_, masses)| masses)
1422}
1423
1424/// THE single assembly source: walk every (scale, outer-net center) local
1425/// residual block exactly once and scatter it into `n_forms` accumulators
1426/// with caller-chosen scalar weights. The energy, its (s, α) jets, and the
1427/// per-scale spectrum are all this routine with different weight closures,
1428/// so a value/derivative desync is structurally impossible.
1429///
1430/// Per block the closure receives `(scale_idx, eps, q, base)` where `q` is
1431/// the truncated kernel sum used by the local residual and `base`
1432/// is the fully-assembled outer weight
1433/// `log_step · ε^(−η) · net_mass_i · q^(1−2α)`, with
1434/// `η = 2s + d(2−2α)` for the available dimension parameter, and writes, per requested
1435/// form, one weight triple `[w_R, w_2, w_3]`. Only `w_R` is live:
1436/// `R = CᵀWC − B·G⁺·Bᵀ/q`, with `G⁺` the rank-revealing pseudo-inverse.
1437/// The extra slots are retained for the ψ layout and receive zero local
1438/// channels because τ no longer changes the energy.
1439///
1440/// The outer sum over centers is coarsened per scale to a deterministic
1441/// ε/2-net with nearest-member mass aggregation (the outer Riemann sum needs
1442/// resolution ε, not the center-spacing floor), so each scale's cost sits at
1443/// its own level and the band totals ~O(m²·d) instead of O(L·m³). The inner
1444/// (local-fit) quadrature always uses the full center set, so the local
1445/// residual identities (exact constant annihilation, PSD) are untouched.
1446pub(crate) fn assemble_weighted_forms<F>(
1447    centers: ArrayView2<'_, f64>,
1448    masses: ArrayView1<'_, f64>,
1449    band: &MeasureJetBand,
1450    order_s: f64,
1451    alpha: f64,
1452    tau0: f64,
1453    n_forms: usize,
1454    channels: usize,
1455    weights: &F,
1456) -> Result<Vec<Array2<f64>>, BasisError>
1457where
1458    F: Fn(usize, f64, f64, f64, &mut [[f64; 3]]) + Sync,
1459{
1460    let m = centers.nrows();
1461    let d = centers.ncols();
1462    if n_forms == 0 || !(1..=3).contains(&channels) {
1463        crate::bail_invalid_basis!(
1464            "measure-jet assembly needs at least one output form and 1..=3 block channels"
1465        );
1466    }
1467    if masses.len() != m {
1468        crate::bail_dim_basis!(
1469            "measure-jet energy mass/center mismatch: {} masses for {} centers",
1470            masses.len(),
1471            m
1472        );
1473    }
1474    if band.eps.is_empty() || band.eps.iter().any(|e| !(e.is_finite() && *e > 0.0)) {
1475        crate::bail_invalid_basis!("measure-jet energy needs a nonempty positive scale band");
1476    }
1477    if !(order_s.is_finite() && order_s > 0.0 && order_s < 2.0) {
1478        crate::bail_invalid_basis!(
1479            "measure-jet order s must lie in (0, 2) for the affine-jet energy; got {order_s}"
1480        );
1481    }
1482    if !(alpha.is_finite() && tau0.is_finite() && tau0 >= 0.0) {
1483        crate::bail_invalid_basis!(
1484            "measure-jet energy needs finite alpha and finite tau0 >= 0; got alpha={alpha}, tau0={tau0}"
1485        );
1486    }
1487    if masses.iter().any(|v| !(v.is_finite() && *v >= 0.0)) {
1488        crate::bail_invalid_basis!("measure-jet energy needs finite nonnegative center masses");
1489    }
1490    let dist2 = pairwise_sq_dists(centers, centers);
1491
1492    // One block of `n_forms` m×m accumulators per scale. Each scale's center
1493    // loop is sequential and the cross-scale sum below runs in band order,
1494    // so the result is bit-deterministic whether or not the scales
1495    // themselves run in parallel.
1496    let assemble_scale = |scale_idx: usize, eps: f64| -> Result<Vec<Array2<f64>>, BasisError> {
1497        let mut out: Vec<Array2<f64>> =
1498            (0..n_forms).map(|_| Array2::<f64>::zeros((m, m))).collect();
1499        let cutoff2 = (MEASURE_JET_PROFILE_CUTOFF * eps) * (MEASURE_JET_PROFILE_CUTOFF * eps);
1500        let inv_two_eps2 = 1.0 / (2.0 * eps * eps);
1501        let eta = 2.0 * order_s + (d as f64) * (2.0 - 2.0 * alpha);
1502        let scale_weight = band.log_step * eps.powf(-eta);
1503        // Outer-quadrature coarsening: greedy ε/2-net over the centers in
1504        // fixed index order (deterministic), with every center's mass
1505        // aggregated to its nearest net member (lowest-index tie break).
1506        let net_radius2 = 0.25 * eps * eps;
1507        let mut outer: Vec<usize> = Vec::new();
1508        for i in 0..m {
1509            if masses[i] <= 0.0 {
1510                continue;
1511            }
1512            let covered = outer.iter().any(|&o| dist2[(i, o)] <= net_radius2);
1513            if !covered {
1514                outer.push(i);
1515            }
1516        }
1517        let mut net_mass = vec![0.0_f64; m];
1518        for i in 0..m {
1519            if masses[i] <= 0.0 {
1520                continue;
1521            }
1522            let mut best = f64::INFINITY;
1523            let mut best_o = usize::MAX;
1524            for &o in &outer {
1525                if dist2[(i, o)] < best {
1526                    best = dist2[(i, o)];
1527                    best_o = o;
1528                }
1529            }
1530            if best_o != usize::MAX {
1531                net_mass[best_o] += masses[i];
1532            }
1533        }
1534        let mut wbuf = vec![[0.0_f64; 3]; n_forms];
1535        for &i in &outer {
1536            // Local neighbor set (always includes i itself).
1537            let mut idx: Vec<usize> = Vec::new();
1538            for j in 0..m {
1539                if dist2[(i, j)] <= cutoff2 {
1540                    idx.push(j);
1541                }
1542            }
1543            let ml = idx.len();
1544            // Kernel weights and mass.
1545            let mut w = Array1::<f64>::zeros(ml);
1546            let mut q = 0.0_f64;
1547            for (a, &j) in idx.iter().enumerate() {
1548                let wj = masses[j] * (-dist2[(i, j)] * inv_two_eps2).exp();
1549                w[a] = wj;
1550                q += wj;
1551            }
1552            if !(q > 0.0) {
1553                continue;
1554            }
1555            // Scaled local features Φ (ml × d) and weighted column means a.
1556            let mut phi = Array2::<f64>::zeros((ml, d));
1557            for (a, &j) in idx.iter().enumerate() {
1558                for k in 0..d {
1559                    phi[(a, k)] = (centers[(j, k)] - centers[(i, k)]) / eps;
1560                }
1561            }
1562            let a_mean = phi.t().dot(&w) / q;
1563            // B = WΦ − w·aᵀ and G = (ΦᵀWΦ)/q − a·aᵀ.
1564            let mut wphi = phi.clone();
1565            for (a, mut row) in wphi.outer_iter_mut().enumerate() {
1566                row.mapv_inplace(|v| v * w[a]);
1567            }
1568            let mut b = wphi.clone();
1569            for (a, mut row) in b.outer_iter_mut().enumerate() {
1570                for k in 0..d {
1571                    row[k] -= w[a] * a_mean[k];
1572                }
1573            }
1574            let mut g = phi.t().dot(&wphi);
1575            g.mapv_inplace(|v| v / q);
1576            for r in 0..d {
1577                for c in 0..d {
1578                    g[(r, c)] -= a_mean[r] * a_mean[c];
1579                }
1580            }
1581            let g_pinv = symmetric_pseudoinverse(&g, "local affine Gram")?;
1582            let bm = b.dot(&g_pinv);
1583            let base = scale_weight * net_mass[i] * q.powf(1.0 - 2.0 * alpha);
1584            weights(scale_idx, eps, q, base, &mut wbuf);
1585            // Scatter-add Σ_k wbuf[k]·R into each form. The τ channels are
1586            // zero because the exact projection is τ-independent.
1587            for (a, &ja) in idx.iter().enumerate() {
1588                let bma = bm.row(a);
1589                for (c, &jc) in idx.iter().enumerate() {
1590                    let b_c = b.row(c);
1591                    let mut val_r = -w[a] * w[c] / q - bma.dot(&b_c) / q;
1592                    if a == c {
1593                        val_r += w[a];
1594                    }
1595                    for (k, out_k) in out.iter_mut().enumerate() {
1596                        let wk = wbuf[k];
1597                        out_k[(ja, jc)] += wk[0] * val_r;
1598                    }
1599                }
1600            }
1601        }
1602        Ok(out)
1603    };
1604
1605    let n_scales = band.eps.len();
1606    let parallel_ok = m
1607        .saturating_mul(m)
1608        .saturating_mul(n_scales)
1609        .saturating_mul(n_forms)
1610        <= MEASURE_JET_PARALLEL_FORM_BUDGET_DOUBLES;
1611    let per_scale: Vec<Vec<Array2<f64>>> = if parallel_ok {
1612        band.eps
1613            .par_iter()
1614            .enumerate()
1615            .map(|(scale_idx, &eps)| assemble_scale(scale_idx, eps))
1616            .collect::<Result<Vec<_>, BasisError>>()?
1617    } else {
1618        band.eps
1619            .iter()
1620            .enumerate()
1621            .map(|(scale_idx, &eps)| assemble_scale(scale_idx, eps))
1622            .collect::<Result<Vec<_>, BasisError>>()?
1623    };
1624
1625    let mut totals: Vec<Array2<f64>> = (0..n_forms).map(|_| Array2::<f64>::zeros((m, m))).collect();
1626    for scale_forms in per_scale {
1627        for (total, part) in totals.iter_mut().zip(scale_forms) {
1628            *total += &part;
1629        }
1630    }
1631    // Numerical symmetrization (every analytic form here is symmetric).
1632    Ok(totals.into_iter().map(|t| (&t + &t.t()) * 0.5).collect())
1633}
1634
1635/// The multiscale jet-residual energy `Q` (m × m, symmetric PSD) on the
1636/// center set. See the module docs for the formula and contracts; the local
1637/// residual form is assembled through the closed-form identities
1638///
1639/// ```text
1640///   CᵀWC          = W − w·wᵀ/q,
1641///   B = CᵀWΦ̃     = WΦ − w·aᵀ          (a = Φᵀw/q),
1642///   G = Φ̃ᵀWΦ̃/q  = (ΦᵀWΦ)/q − a·aᵀ,
1643///   R_loc         = CᵀWC − B·G⁺·Bᵀ/q,
1644/// ```
1645///
1646/// with `G⁺` realized through the symmetric eigendecomposition and a
1647/// machine-precision rank cutoff. One walk of `assemble_weighted_forms`
1648/// with the unit weight.
1649pub fn measure_jet_energy_form(
1650    centers: ArrayView2<'_, f64>,
1651    masses: ArrayView1<'_, f64>,
1652    band: &MeasureJetBand,
1653    order_s: f64,
1654    alpha: f64,
1655    tau0: f64,
1656) -> Result<Array2<f64>, BasisError> {
1657    let mut forms = assemble_weighted_forms(
1658        centers,
1659        masses,
1660        band,
1661        order_s,
1662        alpha,
1663        tau0,
1664        1,
1665        1,
1666        &|_, _, _, base, out: &mut [[f64; 3]]| out[0] = [base, 0.0, 0.0],
1667    )?;
1668    let q = forms.swap_remove(0);
1669    // The energy `Q = Σ wᵢ Rᵢ` is a nonnegative combination of analytically
1670    // PSD local residual forms, so it is PSD in exact arithmetic. The affine
1671    // span is annihilated to machine zero, where roundoff in the per-block
1672    // pseudo-inverse and the centering cancellation leaves the smallest
1673    // eigenvalue at ±ε_mach·‖Q‖. Project onto the PSD cone (floor negative
1674    // eigenvalues at 0) so `vᵀQv ≥ 0` holds exactly for every `v`, including
1675    // the affine directions the energy must annihilate.
1676    project_symmetric_psd(q, "measure-jet energy form")
1677}
1678
1679/// Project a symmetric matrix onto the PSD cone by flooring its negative
1680/// eigenvalues at 0. Only sub-machine-precision negative eigenvalues are
1681/// expected here (the form is analytically PSD); a meaningfully negative
1682/// eigenvalue would indicate an assembly bug, so it is floored but the
1683/// reconstruction otherwise preserves the spectrum exactly.
1684pub(crate) fn project_symmetric_psd(
1685    a: Array2<f64>,
1686    label: &str,
1687) -> Result<Array2<f64>, BasisError> {
1688    let n = a.nrows();
1689    if n == 0 {
1690        return Ok(a);
1691    }
1692    let (evals, evecs) = a.eigh(Side::Lower).map_err(|e| {
1693        BasisError::InvalidInput(format!(
1694            "measure-jet PSD projection `{label}` eigendecomposition failed: {e}"
1695        ))
1696    })?;
1697    if evals.iter().all(|&lam| lam >= 0.0) {
1698        return Ok(a);
1699    }
1700    let mut scaled = evecs.clone();
1701    for (k, mut col) in scaled.axis_iter_mut(Axis(1)).enumerate() {
1702        let lam = evals[k].max(0.0);
1703        col.mapv_inplace(|v| v * lam);
1704    }
1705    let psd = scaled.dot(&evecs.t());
1706    Ok((&psd + &psd.t()) * 0.5)
1707}
1708
1709/// The energy together with its exact first and second jets in the live
1710/// dials, plus zero slots for the retained `ψ_τ = ln τ` coordinate. With
1711/// `g_s = −2 ln ε`, `g_α = −2 ln q`:
1712///
1713/// ```text
1714///   ∂Q/∂s   = Σ g_s·w·R,        ∂²Q/∂s²   = Σ g_s²·w·R,
1715///   ∂Q/∂α   = Σ g_α·w·R,        ∂²Q/∂α²   = Σ g_α²·w·R,
1716///   ∂²Q/∂s∂α = Σ g_s·g_α·w·R,
1717///   ∂Q/∂ψ_τ = ∂²Q/∂ψ_τ² = ∂²Q/∂s∂ψ_τ = ∂²Q/∂α∂ψ_τ = 0.
1718/// ```
1719///
1720/// all scattered from the SAME local blocks as `Q` in one pass (no second
1721/// assembly that could drift). FD-gated in this module's tests. Requires
1722/// `tau0 > 0` only because the retained coordinate is `ln τ`.
1723pub fn measure_jet_energy_form_with_jets(
1724    centers: ArrayView2<'_, f64>,
1725    masses: ArrayView1<'_, f64>,
1726    band: &MeasureJetBand,
1727    order_s: f64,
1728    alpha: f64,
1729    tau0: f64,
1730) -> Result<MeasureJetEnergyJets, BasisError> {
1731    if !(tau0.is_finite() && tau0 > 0.0) {
1732        crate::bail_invalid_basis!(
1733            "measure-jet jets need tau0 > 0 because the retained τ coordinate is ln τ; got {tau0}"
1734        );
1735    }
1736    let mut forms = assemble_weighted_forms(
1737        centers,
1738        masses,
1739        band,
1740        order_s,
1741        alpha,
1742        tau0,
1743        10,
1744        3,
1745        &|_, eps: f64, q: f64, base: f64, out: &mut [[f64; 3]]| {
1746            let gs = -2.0 * eps.ln();
1747            let intrinsic_dim = centers.ncols() as f64;
1748            let ga = 2.0 * intrinsic_dim * eps.ln() - 2.0 * q.max(f64::MIN_POSITIVE).ln();
1749            out[0] = [base, 0.0, 0.0];
1750            out[1] = [gs * base, 0.0, 0.0];
1751            out[2] = [gs * gs * base, 0.0, 0.0];
1752            out[3] = [ga * base, 0.0, 0.0];
1753            out[4] = [ga * ga * base, 0.0, 0.0];
1754            out[5] = [gs * ga * base, 0.0, 0.0];
1755            out[6] = [0.0, 0.0, 0.0];
1756            out[7] = [0.0, 0.0, 0.0];
1757            out[8] = [0.0, 0.0, 0.0];
1758            out[9] = [0.0, 0.0, 0.0];
1759        },
1760    )?;
1761    let d2q_dalpha_dlogtau = forms.pop().expect("ten assembled forms");
1762    let d2q_ds_dlogtau = forms.pop().expect("ten assembled forms");
1763    let d2q_dlogtau2 = forms.pop().expect("ten assembled forms");
1764    let dq_dlogtau = forms.pop().expect("ten assembled forms");
1765    let d2q_ds_dalpha = forms.pop().expect("ten assembled forms");
1766    let d2q_dalpha2 = forms.pop().expect("ten assembled forms");
1767    let dq_dalpha = forms.pop().expect("ten assembled forms");
1768    let d2q_ds2 = forms.pop().expect("ten assembled forms");
1769    let dq_ds = forms.pop().expect("ten assembled forms");
1770    let q = forms.pop().expect("ten assembled forms");
1771    Ok(MeasureJetEnergyJets {
1772        q,
1773        dq_ds,
1774        d2q_ds2,
1775        dq_dalpha,
1776        d2q_dalpha2,
1777        d2q_ds_dalpha,
1778        dq_dlogtau,
1779        d2q_dlogtau2,
1780        d2q_ds_dlogtau,
1781        d2q_dalpha_dlogtau,
1782    })
1783}
1784
1785/// Per-scale energy decomposition of center values `v`: element ℓ is
1786/// `vᵀ Q_ℓ v`, the detail energy charged at scale `ε_ℓ`. Sums exactly to
1787/// `vᵀQv` (same blocks, one-hot weights) and doubles as the scale spectrum
1788/// diagnostic of the fitted intensity field — where along the band the
1789/// signal lives, and the analytic carrier of `∂/∂s` reweightings.
1790pub fn measure_jet_scale_spectrum(
1791    centers: ArrayView2<'_, f64>,
1792    masses: ArrayView1<'_, f64>,
1793    band: &MeasureJetBand,
1794    order_s: f64,
1795    alpha: f64,
1796    tau0: f64,
1797    values: ArrayView1<'_, f64>,
1798) -> Result<Vec<f64>, BasisError> {
1799    if values.len() != centers.nrows() {
1800        crate::bail_dim_basis!(
1801            "measure-jet scale spectrum needs one value per center: {} values for {} centers",
1802            values.len(),
1803            centers.nrows()
1804        );
1805    }
1806    let forms = measure_jet_energy_forms_per_scale(centers, masses, band, order_s, alpha, tau0)?;
1807    Ok(forms
1808        .iter()
1809        .map(|q_l| values.dot(&q_l.dot(&values)))
1810        .collect())
1811}
1812
1813/// The per-scale energy forms `Q_ℓ` (each m × m, symmetric PSD), with
1814/// `Σ_ℓ Q_ℓ = Q` to the PSD-projection floor (same blocks, one-hot weights).
1815/// These are the spectral-split carriers: emitted as separate penalty
1816/// candidates they let the multi-penalty REML engine learn per-level amplitudes
1817/// λ_ℓ directly — scale adaptivity at ρ-speed with no rebuild and no new
1818/// optimizer code.
1819///
1820/// Each level is projected onto the PSD cone for the same reason the fused
1821/// [`measure_jet_energy_form`] is, and the projection is load-bearing HERE in a
1822/// way it is not there — see the note at the return.
1823pub fn measure_jet_energy_forms_per_scale(
1824    centers: ArrayView2<'_, f64>,
1825    masses: ArrayView1<'_, f64>,
1826    band: &MeasureJetBand,
1827    order_s: f64,
1828    alpha: f64,
1829    tau0: f64,
1830) -> Result<Vec<Array2<f64>>, BasisError> {
1831    let n_scales = band.eps.len();
1832    let forms = assemble_weighted_forms(
1833        centers,
1834        masses,
1835        band,
1836        order_s,
1837        alpha,
1838        tau0,
1839        n_scales,
1840        1,
1841        &|scale_idx, _, _, base, out: &mut [[f64; 3]]| {
1842            for (k, slot) in out.iter_mut().enumerate() {
1843                *slot = if k == scale_idx {
1844                    [base, 0.0, 0.0]
1845                } else {
1846                    [0.0, 0.0, 0.0]
1847                };
1848            }
1849        },
1850    )?;
1851    // PSD cone projection, per level. Every `Q_ℓ` is a NONNEGATIVE combination
1852    // of the same analytically-PSD local residual blocks the fused energy sums,
1853    // so it is PSD in exact arithmetic and only the per-block pseudo-inverse and
1854    // centering cancellation put a ±ε_mach·‖Q_ℓ‖ negative in the spectrum —
1855    // the identical situation `measure_jet_energy_form` floors on the cone.
1856    //
1857    // Skipping it here was NOT symmetric with the fused path, because the fused
1858    // path normalizes ONE matrix while the builder normalizes EVERY LEVEL BY ITS
1859    // OWN Frobenius scale. A level whose detail energy is numerically dead
1860    // carries only that roundoff, and dividing roundoff by its own tiny norm
1861    // rescales it to unit norm: a ±ε_mach relative negative becomes an O(1)
1862    // absolute one. `ConstructiveQuadratic::try_from_dense_psd` then rejects the
1863    // candidate and the whole multiscale BUILD fails — measured as
1864    // `IndefinitePenalty { context: "measure-jet scale penalty",
1865    // min_eigenvalue: -0.3039, tolerance: 1.486e-8 }`, where the tolerance is
1866    // √ε_mach against a max |λ| of ~1, i.e. the certified matrix is already
1867    // unit-normalized and the negative is 2e7× tolerance. No real detail
1868    // spectrum is 30% negative; that is normalized roundoff.
1869    //
1870    // Flooring restores the invariant the signature documents, leaves a dead
1871    // level as an exact-zero candidate for `filter_penalty_candidates`/REML to
1872    // deselect rather than a fatal build error, and preserves `Σ_ℓ Q_ℓ = Q` to
1873    // the same machine-precision floor the fused projection already accepts. It
1874    // also stops `measure_jet_scale_spectrum` from being able to report a
1875    // NEGATIVE detail energy `vᵀQ_ℓv`.
1876    forms
1877        .into_iter()
1878        .enumerate()
1879        .map(|(level, q_l)| {
1880            project_symmetric_psd(q_l, &format!("measure-jet per-scale energy form {level}"))
1881        })
1882        .collect()
1883}
1884
1885/// The support diagnostic `ε ↦ q_ε(x★)`: kernel mass of the (frozen) center
1886/// quadrature seen from each query point at every band scale (n_query × L).
1887/// A query ON the web sees its strand's mass already at fine scales; a query
1888/// OFF the web accumulates mass only once ε reaches its distance to the
1889/// support. This is the on-web-ness statistic shipped alongside predictions
1890/// — smooth, multiresolution, derived from the measure with no neighbor
1891/// sets.
1892pub fn measure_jet_support_curve(
1893    queries: ArrayView2<'_, f64>,
1894    centers: ArrayView2<'_, f64>,
1895    masses: ArrayView1<'_, f64>,
1896    eps_band: &[f64],
1897) -> Result<Array2<f64>, BasisError> {
1898    if queries.ncols() != centers.ncols() {
1899        crate::bail_dim_basis!(
1900            "measure-jet support curve dimension mismatch: queries d={} centers d={}",
1901            queries.ncols(),
1902            centers.ncols()
1903        );
1904    }
1905    if masses.len() != centers.nrows() {
1906        crate::bail_dim_basis!(
1907            "measure-jet support curve mass/center mismatch: {} masses for {} centers",
1908            masses.len(),
1909            centers.nrows()
1910        );
1911    }
1912    if eps_band.is_empty() || eps_band.iter().any(|e| !(e.is_finite() && *e > 0.0)) {
1913        crate::bail_invalid_basis!("measure-jet support curve needs a nonempty positive band");
1914    }
1915    validate_finite_points(queries, "queries")?;
1916    validate_finite_points(centers, "centers")?;
1917    let nq = queries.nrows();
1918    let nl = eps_band.len();
1919    // Distances once (GEMM), then every band scale reads the same d² row —
1920    // an L-fold saving over per-scale distance recomputation.
1921    let d2 = pairwise_sq_dists(queries, centers);
1922    let mut out = Array2::<f64>::zeros((nq, nl));
1923    out.axis_iter_mut(Axis(0))
1924        .into_par_iter()
1925        .enumerate()
1926        .for_each(|(qi, mut row)| {
1927            let d2_row = d2.row(qi);
1928            for (li, &eps) in eps_band.iter().enumerate() {
1929                let inv_two_eps2 = 1.0 / (2.0 * eps * eps);
1930                let mut acc = 0.0_f64;
1931                for (j, &dd) in d2_row.iter().enumerate() {
1932                    acc += masses[j] * (-dd * inv_two_eps2).exp();
1933                }
1934                row[li] = acc;
1935            }
1936        });
1937    Ok(out)
1938}
1939
1940pub(crate) fn measure_jet_support_means(
1941    centers: ArrayView2<'_, f64>,
1942    masses: ArrayView1<'_, f64>,
1943    eps_band: &[f64],
1944) -> Result<Vec<f64>, BasisError> {
1945    let total_mass = masses.sum();
1946    if !(total_mass.is_finite() && total_mass > 0.0) {
1947        crate::bail_invalid_basis!(
1948            "measure-jet support means need positive finite total mass; got {total_mass}"
1949        );
1950    }
1951    let support = measure_jet_support_curve(centers, centers, masses, eps_band)?;
1952    let mut means = vec![0.0_f64; eps_band.len()];
1953    for (i, row) in support.rows().into_iter().enumerate() {
1954        let mass = masses[i];
1955        for (mean, &q) in means.iter_mut().zip(row.iter()) {
1956            *mean += mass * q;
1957        }
1958    }
1959    for mean in &mut means {
1960        *mean /= total_mass;
1961        if !(*mean).is_finite() || *mean <= 0.0 {
1962            crate::bail_invalid_basis!(
1963                "measure-jet support mean must be positive and finite; got {mean}"
1964            );
1965        }
1966    }
1967    Ok(means)
1968}
1969
1970/// Gaussian representer features `exp(−‖x − c‖²/(2ℓ²))` (n × m).
1971pub fn measure_jet_design_matrix(
1972    data: ArrayView2<'_, f64>,
1973    centers: ArrayView2<'_, f64>,
1974    length_scale: f64,
1975) -> Result<Array2<f64>, BasisError> {
1976    if data.ncols() != centers.ncols() {
1977        crate::bail_dim_basis!(
1978            "measure-jet design dimension mismatch: data d={} centers d={}",
1979            data.ncols(),
1980            centers.ncols()
1981        );
1982    }
1983    if !(length_scale.is_finite() && length_scale > 0.0) {
1984        crate::bail_invalid_basis!(
1985            "measure-jet design needs a positive finite length_scale; got {length_scale}"
1986        );
1987    }
1988    validate_finite_points(data, "data")?;
1989    validate_finite_points(centers, "centers")?;
1990    let inv_two_l2 = 1.0 / (2.0 * length_scale * length_scale);
1991    // One GEMM for every distance, then the Gaussian applied in place — the
1992    // n×m allocation IS the output, no transient copy.
1993    let mut out = pairwise_sq_dists(data, centers);
1994    out.axis_iter_mut(Axis(0))
1995        .into_par_iter()
1996        .for_each(|mut row| {
1997            row.mapv_inplace(|d2| (-d2 * inv_two_l2).exp());
1998        });
1999    Ok(out)
2000}
2001
2002/// Exact first and diagonal-second derivatives of the Gaussian representer
2003/// design with respect to `u = ln ℓ`.
2004fn measure_jet_design_log_length_jets(
2005    data: ArrayView2<'_, f64>,
2006    centers: ArrayView2<'_, f64>,
2007    length_scale: f64,
2008) -> Result<(Array2<f64>, Array2<f64>), BasisError> {
2009    let kernel = measure_jet_design_matrix(data, centers, length_scale)?;
2010    let squared_distances = pairwise_sq_dists(data, centers);
2011    let inv_l2 = 1.0 / (length_scale * length_scale);
2012    let mut first = kernel.clone();
2013    let mut second = kernel;
2014    for ((first_value, second_value), &distance_squared) in first
2015        .iter_mut()
2016        .zip(second.iter_mut())
2017        .zip(squared_distances.iter())
2018    {
2019        let a = distance_squared * inv_l2;
2020        let kernel_value = *first_value;
2021        *first_value = kernel_value * a;
2022        *second_value = kernel_value * (a * a - 2.0 * a);
2023    }
2024    Ok((first, second))
2025}
2026
2027/// Rank-revealing ambient-linear head lift `T` (d × head_rank) for the
2028/// extrapolation null space (#1845).
2029///
2030/// The measure-jet energy annihilates ambient-affine functions EXACTLY (the
2031/// no-mass contract), so the affine functions are the penalty's null space —
2032/// the directions the fit is free to extend across a training gap. But the
2033/// Gaussian representer design cannot REPRESENT a global affine function off
2034/// its support: a finite sum of decaying bumps reverts to the parametric
2035/// backbone away from the centers, so in a gap the fit collapses toward the
2036/// training mean instead of carrying the flank-attested trend. Completing the
2037/// smoothing-spline structure, the builder appends this ambient-linear null
2038/// space to the design as an UNPENALIZED head (the `{x_1..x_d}` head the frame
2039/// notes §1 pin as the property the representer basis lacked).
2040///
2041/// The head is data-derived and magic-free. Ambient coordinates of data on a
2042/// low intrinsic-dimension stratum are rank-deficient as linear trends, so the
2043/// coordinate columns are orthonormalized on the centers and the
2044/// numerically-degenerate directions dropped. Working in the mean-CENTERED
2045/// coordinate columns (the mass-weighted mean is the intercept's, not the
2046/// head's) makes the rank test measure the genuine spread of the centers along
2047/// each direction rather than its offset; the relative floor
2048/// `MEASURE_JET_PSEUDOINVERSE_RTOL` is the module's own numerical rank
2049/// tolerance (the same one the local Gram pseudo-inverses use). The returned
2050/// `T` satisfies `linear_head(points) = points · T` (the mean-centering only
2051/// informs the keep/drop decision). `T` is a deterministic function of the
2052/// frozen centers + masses, so the frozen replay path reconstructs the
2053/// identical head with no persisted state.
2054///
2055/// This is the LINEAR half of the null space. The realized head block is the
2056/// whole affine null space `[1 | points·T]`; build it through
2057/// [`measure_jet_affine_head_lift`] + [`measure_jet_affine_head_block`], which
2058/// is what the design, the gauge and the null-component penalty all use. A
2059/// linear-only head is a defect, not an economy: the global parametric
2060/// orthogonalization removes ONE design direction, and if the term's null space
2061/// has no constant to give up, the direction it takes comes out of the null
2062/// space itself, leaving `d − 1` free linear directions instead of `d` (#2751).
2063pub fn measure_jet_affine_head_transform(
2064    centers: ArrayView2<'_, f64>,
2065    masses: ArrayView1<'_, f64>,
2066) -> Array2<f64> {
2067    let m = centers.nrows();
2068    let d = centers.ncols();
2069    let total_mass = masses.sum();
2070    // Mass inner product on center values.
2071    let mdot = |u: &Array1<f64>, v: &Array1<f64>| -> f64 {
2072        let mut acc = 0.0;
2073        for i in 0..m {
2074            acc += masses[i] * u[i] * v[i];
2075        }
2076        acc
2077    };
2078    // Mean-centered coordinate columns: the mass-weighted mean is removed so the
2079    // residual mass-norm is the genuine spread of the centers along a direction,
2080    // not dominated by the coordinate's offset (which the intercept owns).
2081    let cols: Vec<Array1<f64>> = (0..d)
2082        .map(|k| {
2083            let col = centers.column(k).to_owned();
2084            let mean = if total_mass > 0.0 {
2085                mdot(&col, &Array1::ones(m)) / total_mass
2086            } else {
2087                0.0
2088            };
2089            col.mapv(|x| x - mean)
2090        })
2091        .collect();
2092    // Relative numerical rank floor from the centered coordinate-column scale.
2093    let max_norm = cols
2094        .iter()
2095        .fold(0.0_f64, |acc, c| acc.max(mdot(c, c).sqrt()));
2096    let drop_below =
2097        (MEASURE_JET_PSEUDOINVERSE_RTOL * (d.max(1) as f64) * max_norm).max(f64::MIN_POSITIVE);
2098    // Mass-weighted modified Gram–Schmidt on the centered columns; `t`
2099    // accumulates the lift in the ORIGINAL coordinate basis, so every kept head
2100    // column is `points · t_r` (up to the intercept-owned constant).
2101    let mut q_cols: Vec<Array1<f64>> = Vec::new();
2102    let mut t_cols: Vec<Array1<f64>> = Vec::new();
2103    for k in 0..d {
2104        let mut v = cols[k].clone();
2105        let mut t = Array1::<f64>::zeros(d);
2106        t[k] = 1.0;
2107        for (q, tq) in q_cols.iter().zip(t_cols.iter()) {
2108            let proj = mdot(q, &v);
2109            v.scaled_add(-proj, q);
2110            t.scaled_add(-proj, tq);
2111        }
2112        let norm = mdot(&v, &v).sqrt();
2113        if norm > drop_below {
2114            v.mapv_inplace(|x| x / norm);
2115            t.mapv_inplace(|x| x / norm);
2116            q_cols.push(v);
2117            t_cols.push(t);
2118        }
2119    }
2120    let head_rank = t_cols.len();
2121    let mut t_mat = Array2::<f64>::zeros((d, head_rank));
2122    for (r, t) in t_cols.into_iter().enumerate() {
2123        t_mat.column_mut(r).assign(&t);
2124    }
2125    t_mat
2126}
2127
2128/// Affine head lift `T_aff` (`(d+1) × (1 + head_rank)`) acting on the augmented
2129/// point rows `[1 | x]`: column 0 is the constant, the rest are the supported
2130/// ambient-linear directions of [`measure_jet_affine_head_transform`].
2131///
2132/// This — not the linear lift alone — is the energy's null space. The energy
2133/// annihilates every AFFINE function of the centers exactly, constant included
2134/// (`affine_function_nullspace_form` projects onto exactly this span), so the
2135/// design block that carries the null space has to span the same thing.
2136///
2137/// The constant column looks redundant against the model intercept and is not.
2138/// The term-collection chokepoint residualizes every measure-jet design against
2139/// the parametric block and reparameterizes to `Z = null(1ᵀX)`, which removes
2140/// exactly one coefficient direction. The null space of the constrained penalty
2141/// is `{γ : Zγ ∈ null(S)}`, so that removal is charged to the null space unless
2142/// the null space contains the constraint's own direction. With a linear-only
2143/// head the term's null space is `span{x·T}`, the constant is nowhere in it,
2144/// and the centering deletes a LINEAR direction: on a 2-D fixture the surviving
2145/// direction is the accidental one with zero data-mean, and every REML fit that
2146/// selects a large energy λ collapses onto it (#2751, measured at Pearson
2147/// 0.705 = |cos 45°| against a planted `x1` plane). With the constant present
2148/// the centering consumes the constant — which the intercept re-supplies —
2149/// and all `head_rank` linear directions stay free. That is exactly how the
2150/// thin-plate/Duchon null space `{1, x_1..x_d}` behaves at the same chokepoint.
2151pub fn measure_jet_affine_head_lift(
2152    centers: ArrayView2<'_, f64>,
2153    masses: ArrayView1<'_, f64>,
2154) -> Array2<f64> {
2155    let linear = measure_jet_affine_head_transform(centers, masses);
2156    let d = centers.ncols();
2157    let mut lift = Array2::<f64>::zeros((d + 1, linear.ncols() + 1));
2158    lift[(0, 0)] = 1.0;
2159    lift.slice_mut(ndarray::s![1.., 1..]).assign(&linear);
2160    lift
2161}
2162
2163/// Realize the affine head block `[1 | points] · T_aff` for the lift returned
2164/// by [`measure_jet_affine_head_lift`]. A zero-column lift (multiscale mode,
2165/// which carries no head) yields a zero-column block.
2166pub fn measure_jet_affine_head_block(
2167    points: ArrayView2<'_, f64>,
2168    lift: ArrayView2<'_, f64>,
2169) -> Array2<f64> {
2170    let n = points.nrows();
2171    let width = lift.ncols();
2172    if width == 0 {
2173        return Array2::<f64>::zeros((n, 0));
2174    }
2175    let d = points.ncols();
2176    assert_eq!(
2177        lift.nrows(),
2178        d + 1,
2179        "affine head lift must have d+1 rows for d ambient coordinates"
2180    );
2181    let mut augmented = Array2::<f64>::ones((n, d + 1));
2182    augmented.slice_mut(ndarray::s![.., 1..]).assign(&points);
2183    augmented.dot(&lift)
2184}
2185
2186/// Resolve the realized representer range ℓ. An explicit positive
2187/// `spec_length_scale` is used verbatim; the `0.0` sentinel auto-initializes
2188/// from the median nearest-center spacing (one spacing width: neighbors
2189/// overlap at exp(−1/2) ≈ 0.61, smooth blend without collinearity).
2190pub fn realized_measure_jet_length_scale(
2191    centers: ArrayView2<'_, f64>,
2192    spec_length_scale: f64,
2193) -> Result<f64, BasisError> {
2194    if spec_length_scale.is_finite() && spec_length_scale > 0.0 {
2195        return Ok(spec_length_scale);
2196    }
2197    if spec_length_scale != 0.0 {
2198        crate::bail_invalid_basis!(
2199            "measure-jet length_scale must be positive (or 0.0 for auto); got {spec_length_scale}"
2200        );
2201    }
2202    let dist2 = pairwise_sq_dists(centers, centers);
2203    let spacing = median_nearest_center_spacing(&dist2)?;
2204    Ok(MEASURE_JET_AUTO_LENGTH_SCALE_FACTOR * spacing)
2205}
2206
2207/// The realized, ψ-FIXED geometry shared by the basis builder and the
2208/// ψ-derivative producer — ONE realization source, so the penalty the fit
2209/// uses and the penalty the ψ-channel differentiates can never drift apart
2210/// (the #901 desync class, excluded structurally).
2211pub(crate) struct RealizedMeasureJetGeometry {
2212    pub(crate) centers: Array2<f64>,
2213    pub(crate) masses: Array1<f64>,
2214    pub(crate) eps_band: Vec<f64>,
2215    pub(crate) log_step: f64,
2216    pub(crate) length_scale: f64,
2217    /// Assembly order for the energy weights: the realized default in
2218    /// per-level mode (absorbed per candidate by normalization), the
2219    /// explicit value in fused mode.
2220    pub(crate) order_s_eval: f64,
2221    /// Spectral-split mode marker (`order_s == 0.0` sentinel).
2222    pub(crate) per_level: bool,
2223    pub(crate) z: Array2<f64>,
2224    pub(crate) coefficient_gauge: gam_problem::Gauge,
2225    pub(crate) kz: Array2<f64>,
2226    /// Affine head lift `T_aff` ((d+1) × head_width): the energy's null space
2227    /// appended to the representer design (#1845), constant included (#2751).
2228    /// The head columns evaluate as `[1 | points] · T_aff`; empty
2229    /// (`(d+1) × 0`) in multiscale mode, which carries no head. Deterministic
2230    /// in the frozen centers + masses, so predict-time replay rebuilds it
2231    /// verbatim.
2232    pub(crate) head_lift: Array2<f64>,
2233}
2234
2235pub(crate) fn realize_measure_jet_geometry(
2236    data: ArrayView2<'_, f64>,
2237    spec: &MeasureJetBasisSpec,
2238) -> Result<RealizedMeasureJetGeometry, BasisError> {
2239    if data.ncols() == 0 {
2240        crate::bail_invalid_basis!("measure-jet smooth needs at least one feature column");
2241    }
2242    validate_finite_points(data, "data")?;
2243    let seed_centers = select_centers_by_strategy(data, &spec.center_strategy)?;
2244    let m = seed_centers.nrows();
2245    if m < 3 {
2246        return Err(BasisError::InsufficientColumnsForConstraint { found: m });
2247    }
2248    let order_s = if spec.order_s == 0.0 {
2249        MEASURE_JET_DEFAULT_ORDER_S
2250    } else {
2251        spec.order_s
2252    };
2253    // Quadrature realization. Fit path: the realized nodes are the cell
2254    // BARYCENTERS of the seed partition (first-moment-exact lumping of μ —
2255    // see `measure_jet_quadrature_nodes`), so the metadata's `centers` are
2256    // already the realized nodes and the frozen path (predict / ψ-trial,
2257    // `CenterStrategy::UserProvided`) replays them verbatim with the frozen
2258    // masses, band, support anchors, and normalization scales.
2259    let (centers, masses, eps_band, log_step) = match &spec.frozen_quadrature {
2260        Some(frozen) => {
2261            if frozen.masses.len() != m {
2262                crate::bail_dim_basis!(
2263                    "frozen measure-jet quadrature mismatch: {} masses for {} centers",
2264                    frozen.masses.len(),
2265                    m
2266                );
2267            }
2268            if frozen.eps_band.is_empty() {
2269                crate::bail_invalid_basis!("frozen measure-jet quadrature has an empty band");
2270            }
2271            let log_step = if frozen.eps_band.len() >= 2 {
2272                (frozen.eps_band[1] / frozen.eps_band[0]).ln()
2273            } else {
2274                std::f64::consts::LN_2
2275            };
2276            (
2277                seed_centers,
2278                frozen.masses.clone(),
2279                frozen.eps_band.clone(),
2280                log_step,
2281            )
2282        }
2283        None => {
2284            let (nodes, masses) = measure_jet_quadrature_nodes(data, seed_centers.view())?;
2285            let band = measure_jet_band(nodes.view(), spec.num_scales)?;
2286            (nodes, masses, band.eps, band.log_step)
2287        }
2288    };
2289    let length_scale = realized_measure_jet_length_scale(centers.view(), spec.length_scale)?;
2290    // Affine extrapolation head (#1845): the raw center space becomes
2291    // `[ m Gaussian representers | head_width affine columns ]`. The head
2292    // carries the penalty's affine null space explicitly — constant included
2293    // (#2751) — so the fit no longer reverts to the parametric backbone (the
2294    // training mean) across an unsupported gap, and so the collection's
2295    // parametric orthogonalization has the constant to consume instead of a
2296    // linear direction.
2297    // The extrapolation head is the single-scale (fused) gap-bridge path. In
2298    // multiscale mode the per-scale spectral penalties carry their own
2299    // structure and the design stays the pure representer basis (the per-level
2300    // replay + width contracts pin `m − 1` columns), so the head is added only
2301    // when the term is single-scale.
2302    let head_lift = if spec.multiscale {
2303        Array2::<f64>::zeros((centers.ncols() + 1, 0))
2304    } else {
2305        measure_jet_affine_head_lift(centers.view(), masses.view())
2306    };
2307    let head_width = head_lift.ncols();
2308    let m_aug = m + head_width;
2309    let k_cc = measure_jet_design_matrix(centers.view(), centers.view(), length_scale)?;
2310    let head_cc = measure_jet_affine_head_block(centers.view(), head_lift.view());
2311    // Realized-design constraint transform. In single-scale mode the explicit
2312    // affine head and Gaussian representers can otherwise carry the same affine
2313    // CENTER values in two different ways. That is a genuine gauge redundancy,
2314    // not a reason to ridge either coefficient block. At fit time remove it
2315    // exactly by restricting the RBF center values to the mass-orthogonal
2316    // complement of the supported affine space:
2317    //
2318    //   C = A^T W K_cc,       Z_rbf = null(C).
2319    //
2320    // The head then passes through as an identity block. The frozen composed
2321    // `z · z_parametric` is replayed verbatim at prediction/ψ trials (#532), so
2322    // the rank-revealed section never changes after fit-time realization. In
2323    // multiscale mode there is no explicit head, hence no affine duplication;
2324    // retain the existing representer sum-to-zero section there.
2325    let (z, coefficient_gauge) = match &spec.identifiability {
2326        MeasureJetIdentifiability::FrozenTransform { transform } => {
2327            if transform.nrows() != m_aug {
2328                crate::bail_dim_basis!(
2329                    "frozen measure-jet identifiability transform mismatch: {} representers + {} head columns but transform has {} rows",
2330                    m,
2331                    head_width,
2332                    transform.nrows()
2333                );
2334            }
2335            (
2336                transform.clone(),
2337                gam_problem::Gauge::from_block_transforms(&[transform.clone()]),
2338            )
2339        }
2340        MeasureJetIdentifiability::CenterSumToZero => {
2341            let z_rbf = if head_width > 0 {
2342                // `head_cc` IS the affine value basis A at the centers, by
2343                // construction (both come from `measure_jet_affine_head_lift`),
2344                // so the gauge constrains the representers against exactly the
2345                // span the head carries.
2346                let mut weighted_affine = head_cc.clone();
2347                for (i, mut row) in weighted_affine.outer_iter_mut().enumerate() {
2348                    row.mapv_inplace(|v| v * masses[i]);
2349                }
2350                // `rrqr_nullspace_basis(B)` returns null(B^T). Here
2351                // `B = K_cc^T W A = C^T`, hence the returned columns span
2352                // null(C), exactly the required RBF coefficient section.
2353                let constraint_cross = k_cc.t().dot(&weighted_affine);
2354                rrqr_nullspace_basis(&constraint_cross, default_rrqr_rank_alpha())
2355                    .map_err(BasisError::LinalgError)?
2356                    .0
2357            } else {
2358                let u = householder_sum_to_zero_u(m);
2359                householder_sum_to_zero_z(&u)
2360            };
2361            let z_rbf = condition_representer_section(&k_cc, &z_rbf)?;
2362            let rbf_rank = z_rbf.ncols();
2363            let mut z_block = Array2::<f64>::zeros((m_aug, rbf_rank + head_width));
2364            z_block
2365                .slice_mut(ndarray::s![..m, ..rbf_rank])
2366                .assign(&z_rbf);
2367            for r in 0..head_width {
2368                z_block[(m + r, rbf_rank + r)] = 1.0;
2369            }
2370            (
2371                z_block.clone(),
2372                gam_problem::Gauge::from_block_transforms(&[z_block]),
2373            )
2374        }
2375    };
2376    // Augmented raw center matrix `[K(centers, centers) | A]`, so the
2377    // restricted `kz` maps constrained coefficients to center nodal values for
2378    // BOTH the representers and the head; the energy annihilates the head block
2379    // (affine) to machine precision, so it stays the unpenalized null space.
2380    let mut k_aug = Array2::<f64>::zeros((m, m_aug));
2381    k_aug.slice_mut(ndarray::s![.., ..m]).assign(&k_cc);
2382    if head_width > 0 {
2383        k_aug.slice_mut(ndarray::s![.., m..]).assign(&head_cc);
2384    }
2385    let kz = coefficient_gauge.restrict_design(&k_aug);
2386    Ok(RealizedMeasureJetGeometry {
2387        centers,
2388        masses,
2389        eps_band,
2390        log_step,
2391        length_scale,
2392        order_s_eval: order_s,
2393        // Multiscale (per-scale spectral) energy is an EXPLICIT opt-in (#1116):
2394        // one Primary energy at any center count unless the spec asks for the
2395        // scale split. The independent null-component candidate is orthogonal
2396        // to this mode decision. No center-count auto-gate.
2397        per_level: spec.multiscale,
2398        z,
2399        coefficient_gauge,
2400        kz,
2401        head_lift,
2402    })
2403}
2404
2405/// Estimate the ambient input-measurement-error scale `σ_coord` — the
2406/// perpendicular off-manifold residual spread of the empirical measure — for
2407/// the errors-in-variables predictive-variance term `Var_input = ∇f̂ᵀΣ_x∇f̂`,
2408/// `Σ_x = σ_coord²·I` (issue #2225).
2409///
2410/// The measure-jet models data concentrated near an unknown low-intrinsic-
2411/// dimension set sampled with isotropic ambient coordinate noise. In a
2412/// neighborhood the set is locally affine, so the noise lives in the ambient
2413/// directions ORTHOGONAL to the local tangent — exactly the smallest principal
2414/// directions of the local data covariance. This is the standard local-PCA
2415/// noise floor: for each center's nearest-assignment cell with enough points to
2416/// span a tangent (`≥ d + 1`, the linear-algebra rank requirement — not a tuned
2417/// knob), the smallest eigenvalue of the cell-local covariance estimates the
2418/// perpendicular variance `σ_coord²`; averaging over cells (weighted by the
2419/// cell count) pools the estimate. No response values, no smoothing dial, and
2420/// no magic constant enter — it is a pure function of the ambient point cloud
2421/// and the frozen centers, in the centers' (standardized) coordinate frame.
2422///
2423/// Returns `None` when no cell can span a tangent (e.g. `d`-dimensional data
2424/// with fewer than `d + 1` points per cell, or a full-dimensional stratum with
2425/// no separable perpendicular direction) — the caller then leaves `Var_input`
2426/// disabled rather than invent a scale.
2427pub fn measure_jet_input_noise_scale(
2428    data: ArrayView2<'_, f64>,
2429    centers: ArrayView2<'_, f64>,
2430) -> Result<Option<f64>, BasisError> {
2431    let d = data.ncols();
2432    let m = centers.nrows();
2433    if d == 0 || m == 0 || data.nrows() == 0 {
2434        return Ok(None);
2435    }
2436    if centers.ncols() != d {
2437        crate::bail_dim_basis!(
2438            "measure-jet input-noise estimate: data d={d} disagrees with centers d={}",
2439            centers.ncols()
2440        );
2441    }
2442    validate_finite_points(data, "data")?;
2443    validate_finite_points(centers, "centers")?;
2444    // Nearest-center assignment (the same rule that lumps the quadrature
2445    // masses): the squared-distance Gram, argmin per row.
2446    let sq = pairwise_sq_dists(data, centers);
2447    let mut members: Vec<Vec<usize>> = vec![Vec::new(); m];
2448    for (j, row) in sq.axis_iter(Axis(0)).enumerate() {
2449        let mut best = 0usize;
2450        let mut best_d = f64::INFINITY;
2451        for (i, &dij) in row.iter().enumerate() {
2452            if dij < best_d {
2453                best_d = dij;
2454                best = i;
2455            }
2456        }
2457        members[best].push(j);
2458    }
2459    let mut weighted_sum = 0.0_f64;
2460    let mut weight = 0.0_f64;
2461    for cell in &members {
2462        let n_i = cell.len();
2463        // A cell needs at least d + 1 points to define a full-rank local
2464        // covariance; otherwise its smallest eigenvalue is a spurious zero.
2465        if n_i < d + 1 {
2466            continue;
2467        }
2468        // Cell-local mean and covariance in ambient coordinates.
2469        let mut mean = Array1::<f64>::zeros(d);
2470        for &j in cell {
2471            mean += &data.row(j);
2472        }
2473        mean /= n_i as f64;
2474        let mut cov = Array2::<f64>::zeros((d, d));
2475        for &j in cell {
2476            let mut centered = data.row(j).to_owned();
2477            centered -= &mean;
2478            for a in 0..d {
2479                for b in 0..d {
2480                    cov[(a, b)] += centered[a] * centered[b];
2481                }
2482            }
2483        }
2484        cov /= n_i as f64;
2485        // Symmetrize against accumulation asymmetry, then read the smallest
2486        // eigenvalue = the perpendicular (noise) principal variance.
2487        let cov_sym = (&cov + &cov.t()) * 0.5;
2488        let (evals, _) = cov_sym.eigh(Side::Lower).map_err(|e| {
2489            BasisError::InvalidInput(format!(
2490                "measure-jet input-noise estimate: local covariance eigendecomposition failed: {e}"
2491            ))
2492        })?;
2493        let smallest = evals
2494            .iter()
2495            .copied()
2496            .fold(f64::INFINITY, |acc, v| acc.min(v))
2497            .max(0.0);
2498        if smallest.is_finite() {
2499            weighted_sum += n_i as f64 * smallest;
2500            weight += n_i as f64;
2501        }
2502    }
2503    if weight <= 0.0 {
2504        return Ok(None);
2505    }
2506    let sigma2 = weighted_sum / weight;
2507    if !(sigma2.is_finite() && sigma2 > 0.0) {
2508        return Ok(None);
2509    }
2510    Ok(Some(sigma2.sqrt()))
2511}
2512
2513/// Whether a measure-jet spec runs in multiscale mode (per-scale spectral
2514/// energies + `(α, ln τ)` ψ dials). The separate `double_penalty`
2515/// affine/null-component candidate is available in both modes. This is the
2516/// single source of truth shared by the builder and outer enrollment predicates,
2517/// so the energy layout and ψ dimension cannot disagree. Multiscale is an
2518/// explicit opt-in (`spec.multiscale`); there is no center-count auto-gate
2519/// (#1116).
2520pub fn measure_jet_multiscale_mode(spec: &MeasureJetBasisSpec) -> bool {
2521    spec.multiscale
2522}
2523
2524/// Build the measure-jet smooth: Gaussian representer design `K(data,
2525/// centers)·z`, multiscale jet-residual penalty (one candidate per scale in
2526/// spectral mode, one Primary in pinned-order mode), an optional separate
2527/// function-space null-component candidate, and the replayable
2528/// [`BasisMetadata::MeasureJet`]. The geometry comes from the
2529/// empirical measure (centers + masses + band) through the shared
2530/// realization helper — the same source the ψ-derivative producer uses.
2531pub fn build_measure_jet_basis(
2532    data: ArrayView2<'_, f64>,
2533    spec: &MeasureJetBasisSpec,
2534) -> Result<BasisBuildResult, BasisError> {
2535    let RealizedMeasureJetGeometry {
2536        centers,
2537        masses,
2538        eps_band,
2539        log_step,
2540        length_scale,
2541        order_s_eval: order_s,
2542        per_level,
2543        z,
2544        coefficient_gauge,
2545        kz,
2546        head_lift,
2547    } = realize_measure_jet_geometry(data, spec)?;
2548    let band = MeasureJetBand {
2549        eps: eps_band.clone(),
2550        log_step,
2551    };
2552    let m = centers.nrows();
2553    let head_width = head_lift.ncols();
2554    let m_aug = m + head_width;
2555    // Augmented raw design `[K(data, centers) | [1 | data]·T_aff]` (#1845): the
2556    // head columns are the AFFINE extrapolation basis, which is the energy's
2557    // whole null space (#2751). The gauge restricts BOTH blocks together, so
2558    // the frozen composed transform replays the head verbatim at predict time.
2559    let kernel_design = measure_jet_design_matrix(data, centers.view(), length_scale)?;
2560    let mut raw_design = Array2::<f64>::zeros((data.nrows(), m_aug));
2561    raw_design
2562        .slice_mut(ndarray::s![.., ..m])
2563        .assign(&kernel_design);
2564    if head_width > 0 {
2565        let head_design = measure_jet_affine_head_block(data, head_lift.view());
2566        raw_design
2567            .slice_mut(ndarray::s![.., m..])
2568            .assign(&head_design);
2569    }
2570    let constrained_design = coefficient_gauge.restrict_design(&raw_design);
2571    let design = gam_linalg::matrix::DesignMatrix::Dense(
2572        gam_linalg::matrix::DenseDesignMatrix::from(constrained_design),
2573    );
2574    let support_means = measure_jet_support_means(centers.view(), masses.view(), &eps_band)?;
2575    // Spectral/geometric split. With the auto order sentinel (order_s == 0.0)
2576    // the term emits one candidate PER scale: the multi-penalty REML engine
2577    // then learns the level amplitudes λ_ℓ directly — scale adaptivity at
2578    // ρ-speed, dead scales REML-deselected (the Duchon-ARD pattern) — and the
2579    // fitted order is read off the spectrum (ŝ = −½ · slope of ln λ̂_ℓ on
2580    // ln ε_ℓ) instead of being optimized. An explicit s > 0 pins the Mellin
2581    // weights and fuses the band into one candidate. The Mellin prefactor
2582    // ε^(−η)·log_step inside each per-scale form is absorbed by the
2583    // per-candidate Frobenius normalization, so REML owns the amplitudes
2584    // outright. The sentinel itself is persisted in the metadata as the mode
2585    // marker: a replay MUST re-enter the same mode or the penalty count
2586    // desyncs (the gam#860 trap class).
2587    let mut candidates = Vec::new();
2588    let mut penalty_normalization_scales = Vec::new();
2589    let mut raw_penalty_normalization_scales = Vec::new();
2590    let mut fused_penalty_normalization_scale = None;
2591    if per_level {
2592        let forms = measure_jet_energy_forms_per_scale(
2593            centers.view(),
2594            masses.view(),
2595            &band,
2596            order_s,
2597            spec.alpha,
2598            spec.tau0,
2599        )?;
2600        for (level, q_l) in forms.into_iter().enumerate() {
2601            // Constructive pullback, not a dense triple product: the per-scale
2602            // form is PSD by construction and so is its pullback, and only the
2603            // arithmetic can lose that (#2761).
2604            let s_l = constructive_pullback_center_form(&kz, &q_l, "measure-jet scale penalty")?;
2605            let c_l = constructive_frobenius_scale(&s_l);
2606            let intrinsic_dim = centers.ncols() as f64;
2607            let eta = 2.0 * order_s + intrinsic_dim * (2.0 - 2.0 * spec.alpha);
2608            let scale_weight = log_step * eps_band[level].powf(-eta);
2609            penalty_normalization_scales.push(c_l);
2610            raw_penalty_normalization_scales.push(c_l / scale_weight);
2611            candidates.push(PenaltyCandidate {
2612                matrix: s_l.scaled(1.0 / c_l, "normalized measure-jet scale penalty")?,
2613                source: PenaltySource::Other(format!("measure_jet_scale_{level}")),
2614                normalization_scale: c_l,
2615                kronecker_factors: None,
2616                op: None,
2617            });
2618        }
2619    } else {
2620        let q_form = measure_jet_energy_form(
2621            centers.view(),
2622            masses.view(),
2623            &band,
2624            order_s,
2625            spec.alpha,
2626            spec.tau0,
2627        )?;
2628        // The Primary is exactly the jet-energy functional pulled back through
2629        // the center evaluation map. It is independent of `double_penalty`:
2630        // statistical selection is a distinct REML component below, never a
2631        // fixed coefficient toll fused into this estimand.
2632        let penalty =
2633            constructive_pullback_center_form(&kz, &q_form, "measure-jet primary penalty")?;
2634        let c_primary = constructive_frobenius_scale(&penalty);
2635        fused_penalty_normalization_scale = Some(c_primary);
2636        // Declare the energy's structural null frame on the shipped Primary
2637        // (#2761, the #2445 mechanism): the affine head is null by theorem, and
2638        // the pullback's NUMERICAL rank falls as the representer range grows, so
2639        // a rank test on the shipped matrix would let a design-moving ℓ decide
2640        // the double-penalty topology between outer trials.
2641        let mut primary =
2642            penalty.scaled(1.0 / c_primary, "normalized measure-jet primary penalty")?;
2643        if let Some(frame) = measure_jet_primary_structural_null_frame(&z, m, head_width)? {
2644            primary = primary.with_structural_null_frame(
2645                frame,
2646                "measure-jet primary structural null declaration",
2647            )?;
2648        }
2649        candidates.push(PenaltyCandidate {
2650            matrix: primary,
2651            source: PenaltySource::Primary,
2652            normalization_scale: c_primary,
2653            kronecker_factors: None,
2654            op: None,
2655        });
2656    }
2657    // Explicit null recovery is a genuine statistical component: penalize the
2658    // affine/null FUNCTION projection under the empirical-measure mass metric,
2659    // and let REML select its strength independently in both modes. This is the
2660    // standard double-penalty decomposition (roughness + null component); no
2661    // coefficient identity and no hard-coded mixture changes the Primary.
2662    if spec.double_penalty {
2663        let null_penalty = affine_function_nullspace_quadratic(&kz, centers.view(), masses.view())?;
2664        let (_, c_null) = normalize_penalty(null_penalty.dense());
2665        candidates.push(PenaltyCandidate {
2666            matrix: null_penalty
2667                .scaled(1.0 / c_null, "normalized measure-jet null-function penalty")?,
2668            source: PenaltySource::DoublePenaltyNullspace,
2669            normalization_scale: c_null,
2670            kronecker_factors: None,
2671            op: None,
2672        });
2673        // Decide the ridge's fate in THIS chart, the way the term-collection
2674        // chokepoint decides it in its own (#2433's repair, which periodic
2675        // Duchon already carries verbatim, extended here for #2761).
2676        //
2677        // The collection applies its global gauge and then rebuilds the ridge
2678        // from `null(Primary_constrained)`; a chart that has taken the last
2679        // structural null direction leaves nothing for the ridge to shrink and
2680        // the collection drops it. A frozen composed chart — which is what
2681        // every outer ψ trial and every predict-time replay rebuilds in — is
2682        // exactly such a chart for a 1-D measure-jet term, where the parametric
2683        // orthogonalization absorbs the whole affine head. Emitting the raw
2684        // ridge there produces a LOCAL topology of 2 against the collection's
2685        // cached 1, and the incremental realizer aborts the outer search with
2686        // `topology changed ... active_penalties=2, cached_penalties=1`.
2687        //
2688        // Running the same rebuild locally makes the two layers agree by
2689        // construction instead of by coincidence. In the cold chart the head is
2690        // still present, the rebuild keeps the ridge, and this is a no-op on
2691        // the shipped topology.
2692        let primary_physical = candidates
2693            .iter()
2694            .find(|candidate| matches!(candidate.source, PenaltySource::Primary))
2695            .map(|candidate| {
2696                candidate.matrix.scaled(
2697                    candidate.normalization_scale,
2698                    "physical measure-jet primary",
2699                )
2700            })
2701            .transpose()?;
2702        if let Some(primary_physical) = primary_physical {
2703            let width = primary_physical.nrows();
2704            for candidate in &mut candidates {
2705                if !matches!(candidate.source, PenaltySource::DoublePenaltyNullspace) {
2706                    continue;
2707                }
2708                let ridge_physical = candidate.matrix.scaled(
2709                    candidate.normalization_scale,
2710                    "physical measure-jet null-function penalty",
2711                )?;
2712                match super::rebuild_metric_consistent_ridge(&primary_physical, &ridge_physical)? {
2713                    Some(rebuilt) => {
2714                        let normalized = super::normalize_constructive_penalty_candidate(
2715                            rebuilt,
2716                            PenaltySource::DoublePenaltyNullspace,
2717                        )?;
2718                        candidate.matrix = normalized.matrix;
2719                        candidate.normalization_scale = normalized.normalization_scale;
2720                    }
2721                    None => {
2722                        candidate.matrix = ConstructiveQuadratic::zero(width);
2723                        candidate.normalization_scale = 1.0;
2724                    }
2725                }
2726                candidate.kronecker_factors = None;
2727                candidate.op = None;
2728            }
2729        }
2730    }
2731    let filtered = filter_penalty_candidates(candidates)?;
2732    // #2225: compute the errors-in-variables input-noise scale while `centers`
2733    // is still owned; it is moved into the metadata `centers` field below.
2734    let sigma_coord = measure_jet_input_noise_scale(data, centers.view())?;
2735    Ok(BasisBuildResult {
2736        design,
2737        affine_offset: None,
2738        active_penalties: filtered.active,
2739        dropped_penalties: filtered.dropped,
2740        metadata: BasisMetadata::MeasureJet {
2741            centers,
2742            input_scale: crate::IsotropicScale::ONE,
2743            // The realized range from `realize_measure_jet_geometry`, in the
2744            // same frame as `centers` and `eps_band`.  Unlike the other three
2745            // Euclidean families the term-collection wrapper does NOT restore
2746            // an original-units value over this, so the standardized tag
2747            // survives to every consumer (#2636).
2748            length_scale: crate::StandardizedUnits::new(length_scale),
2749            eps_band,
2750            // The SPEC's order field, sentinel included: 0.0 marks per-level
2751            // (spectral) mode and must replay as per-level — persisting the
2752            // realized default here would silently flip the rebuild into
2753            // fused mode and desync the penalty count.
2754            order_s: spec.order_s,
2755            alpha: spec.alpha,
2756            tau0: spec.tau0,
2757            masses,
2758            support_means,
2759            penalty_normalization_scales,
2760            raw_penalty_normalization_scales,
2761            fused_penalty_normalization_scale,
2762            constraint_transform: Some(z),
2763            // Perpendicular off-manifold residual scale of the fit rows in the
2764            // centers' frame — the errors-in-variables input-noise scale (#2225).
2765            sigma_coord,
2766        },
2767        kronecker_factored: None,
2768        joint_null_rotation: None,
2769    })
2770}
2771
2772/// Exact ψ-jets of the REALIZED measure-jet penalty candidates, adapted to
2773/// the anisotropic group-ψ carrier the spatial optimizer consumes.
2774///
2775/// Coordinates (the layout contract for the registration arm):
2776/// - per-level (spectral) mode: `[ln ℓ?, α, ln τ]` — order is absorbed by the
2777///   REML-learned scale amplitudes; `ln τ` is retained as an inert coordinate;
2778/// - single-scale mode: `[ln ℓ?]`, because its energy dials are fixed.
2779///
2780/// Only `ln ℓ` moves the design. It also moves every coefficient-space penalty
2781/// pullback through the center evaluation map `E(ℓ)`; `(α, ln τ)` move only the
2782/// per-scale center-value forms. Exact diagonal and mixed product-rule jets are
2783/// emitted before Frobenius normalization.
2784/// Penalty derivatives are routed through the SAME constrained Frobenius
2785/// normalization as the fit-time candidates
2786/// (`normalize_penaltywith_psi_derivatives` + the cross rule), so criterion
2787/// value and criterion derivative share one normalization — the #901 lesson
2788/// made structural. The function-space null candidate has nonzero `ln ℓ` jets
2789/// and zero `(α, ln τ)` jets. The per-candidate layout follows the builder's
2790/// ORIGINAL order (scale candidates or Primary, then null component); consumers
2791/// align to the FITTED penalty list via
2792/// `ActivePenaltyInfo.original_index` when the candidate filter dropped
2793/// any.
2794pub fn build_measure_jet_basis_psi_derivatives(
2795    data: ArrayView2<'_, f64>,
2796    spec: &MeasureJetBasisSpec,
2797) -> Result<AnisoBasisPsiDerivatives, BasisError> {
2798    if !(spec.tau0.is_finite() && spec.tau0 > 0.0) {
2799        crate::bail_invalid_basis!(
2800            "measure-jet ψ derivatives need tau0 > 0 because the retained τ coordinate is ln τ; got {}",
2801            spec.tau0
2802        );
2803    }
2804    let geom = realize_measure_jet_geometry(data, spec)?;
2805    let band = MeasureJetBand {
2806        eps: geom.eps_band.clone(),
2807        log_step: geom.log_step,
2808    };
2809    let n = data.nrows();
2810    let p = geom.kz.ncols();
2811    let m = geom.centers.nrows();
2812    let m_aug = m + geom.head_lift.ncols();
2813
2814    struct LengthScaleJets {
2815        evaluation_first: Array2<f64>,
2816        evaluation_second: Array2<f64>,
2817        design_first: Array2<f64>,
2818        design_second: Array2<f64>,
2819    }
2820
2821    // The Gaussian representer range moves both the FIT design and the center
2822    // evaluation map `E = [K_cc | A_head] Z`. The affine head is ℓ-invariant,
2823    // so its raw derivative columns are exactly zero before applying the frozen
2824    // Gauge section. Keeping `Z` frozen is the replay contract: rank/gauge
2825    // realization happens once at fit time, then every ψ trial differentiates
2826    // the same coefficient chart.
2827    let length_scale_jets = if spec.learn_length_scale {
2828        let (dk_data, d2k_data) =
2829            measure_jet_design_log_length_jets(data, geom.centers.view(), geom.length_scale)?;
2830        let mut dk_data_aug = Array2::<f64>::zeros((n, m_aug));
2831        let mut d2k_data_aug = Array2::<f64>::zeros((n, m_aug));
2832        dk_data_aug.slice_mut(ndarray::s![.., ..m]).assign(&dk_data);
2833        d2k_data_aug
2834            .slice_mut(ndarray::s![.., ..m])
2835            .assign(&d2k_data);
2836
2837        let (dk_centers, d2k_centers) = measure_jet_design_log_length_jets(
2838            geom.centers.view(),
2839            geom.centers.view(),
2840            geom.length_scale,
2841        )?;
2842        let mut dk_centers_aug = Array2::<f64>::zeros((m, m_aug));
2843        let mut d2k_centers_aug = Array2::<f64>::zeros((m, m_aug));
2844        dk_centers_aug
2845            .slice_mut(ndarray::s![.., ..m])
2846            .assign(&dk_centers);
2847        d2k_centers_aug
2848            .slice_mut(ndarray::s![.., ..m])
2849            .assign(&d2k_centers);
2850
2851        Some(LengthScaleJets {
2852            evaluation_first: geom.coefficient_gauge.restrict_design(&dk_centers_aug),
2853            evaluation_second: geom.coefficient_gauge.restrict_design(&d2k_centers_aug),
2854            design_first: geom.coefficient_gauge.restrict_design(&dk_data_aug),
2855            design_second: geom.coefficient_gauge.restrict_design(&d2k_data_aug),
2856        })
2857    } else {
2858        None
2859    };
2860
2861    let coord_offset = usize::from(length_scale_jets.is_some());
2862    let n_coords = coord_offset + if geom.per_level { 2 } else { 0 };
2863    let pairs: Vec<(usize, usize)> = (0..n_coords)
2864        .flat_map(|a| ((a + 1)..n_coords).map(move |b| (a, b)))
2865        .collect();
2866    let zero_p = || Array2::<f64>::zeros((p, p));
2867
2868    struct RawPenaltyJets {
2869        value: Array2<f64>,
2870        first: Vec<Array2<f64>>,
2871        second_diag: Vec<Array2<f64>>,
2872        cross: Vec<Array2<f64>>,
2873    }
2874
2875    let sandwich = |form: &Array2<f64>| pullback_center_form(&geom.kz, form);
2876    let length_diag = |form: &Array2<f64>| {
2877        let jets = length_scale_jets
2878            .as_ref()
2879            .expect("length-scale form jets require an enrolled length coordinate");
2880        pullback_center_form_log_length_jets(
2881            &geom.kz,
2882            &jets.evaluation_first,
2883            &jets.evaluation_second,
2884            form,
2885        )
2886    };
2887    let length_cross = |form_first: &Array2<f64>| {
2888        let jets = length_scale_jets
2889            .as_ref()
2890            .expect("length-scale cross jets require an enrolled length coordinate");
2891        pullback_center_form_log_length_cross(&geom.kz, &jets.evaluation_first, form_first)
2892    };
2893
2894    // Raw (pre-normalization) value + exact jet stacks per ORIGINAL candidate.
2895    // Coordinate order is `[lnℓ?, α, lnτ]` in multiscale mode and `[lnℓ?]`
2896    // in single-scale mode. Candidate order exactly mirrors the value builder:
2897    // scale candidates or Primary first, then the optional null-component
2898    // candidate. Active filtering aligns through `ActivePenaltyInfo::original_index`.
2899    // The single-scale Primary, when there is one. `None` in per-level mode,
2900    // which emits scale candidates instead and therefore never rebuilds the
2901    // null component.
2902    let mut single_scale_primary: Option<ConstructiveQuadratic> = None;
2903    let mut raw: Vec<RawPenaltyJets> = if geom.per_level {
2904        let l_count = band.eps.len();
2905        // Six forms per scale: value, ∂α, ∂α², and zero τ slots — same
2906        // blocks, one walk (single-source rule).
2907        let forms = assemble_weighted_forms(
2908            geom.centers.view(),
2909            geom.masses.view(),
2910            &band,
2911            geom.order_s_eval,
2912            spec.alpha,
2913            spec.tau0,
2914            6 * l_count,
2915            3,
2916            &|scale_idx, eps: f64, q: f64, base: f64, out: &mut [[f64; 3]]| {
2917                for slot in out.iter_mut() {
2918                    *slot = [0.0, 0.0, 0.0];
2919                }
2920                let intrinsic_dim = geom.centers.ncols() as f64;
2921                let ga = 2.0 * intrinsic_dim * eps.ln() - 2.0 * q.max(f64::MIN_POSITIVE).ln();
2922                let k0 = 6 * scale_idx;
2923                out[k0] = [base, 0.0, 0.0];
2924                out[k0 + 1] = [ga * base, 0.0, 0.0];
2925                out[k0 + 2] = [ga * ga * base, 0.0, 0.0];
2926                out[k0 + 3] = [0.0, 0.0, 0.0];
2927                out[k0 + 4] = [0.0, 0.0, 0.0];
2928                out[k0 + 5] = [0.0, 0.0, 0.0];
2929            },
2930        )?;
2931        let alpha_coord = coord_offset;
2932        let tau_coord = coord_offset + 1;
2933        let mut raw = Vec::with_capacity(l_count + usize::from(spec.double_penalty));
2934        for level in 0..l_count {
2935            let chunk = &forms[6 * level..6 * level + 6];
2936            let mut first: Vec<Array2<f64>> = (0..n_coords).map(|_| zero_p()).collect();
2937            let mut second_diag: Vec<Array2<f64>> = (0..n_coords).map(|_| zero_p()).collect();
2938            first[alpha_coord] = sandwich(&chunk[1]);
2939            first[tau_coord] = sandwich(&chunk[3]);
2940            second_diag[alpha_coord] = sandwich(&chunk[2]);
2941            second_diag[tau_coord] = sandwich(&chunk[4]);
2942            if coord_offset == 1 {
2943                let (ell_first, ell_second) = length_diag(&chunk[0]);
2944                first[0] = ell_first;
2945                second_diag[0] = ell_second;
2946            }
2947            let mut cross: Vec<Array2<f64>> = (0..pairs.len()).map(|_| zero_p()).collect();
2948            for (pair_idx, &(a, b)) in pairs.iter().enumerate() {
2949                cross[pair_idx] = if coord_offset == 1 && a == 0 && b == alpha_coord {
2950                    length_cross(&chunk[1])
2951                } else if coord_offset == 1 && a == 0 && b == tau_coord {
2952                    length_cross(&chunk[3])
2953                } else if a == alpha_coord && b == tau_coord {
2954                    sandwich(&chunk[5])
2955                } else {
2956                    zero_p()
2957                };
2958            }
2959            raw.push(RawPenaltyJets {
2960                value: sandwich(&chunk[0]),
2961                first,
2962                second_diag,
2963                cross,
2964            });
2965        }
2966        raw
2967    } else {
2968        // Single-scale mode enrolls no `(s, α, lnτ)` penalty dials. It still
2969        // emits the pure Primary and, when requested, a separate REML null
2970        // component; an opt-in `lnℓ` coordinate differentiates both pullbacks.
2971        let q_form = measure_jet_energy_form(
2972            geom.centers.view(),
2973            geom.masses.view(),
2974            &band,
2975            geom.order_s_eval,
2976            spec.alpha,
2977            spec.tau0,
2978        )?;
2979        let mut first: Vec<Array2<f64>> = (0..n_coords).map(|_| zero_p()).collect();
2980        let mut second_diag: Vec<Array2<f64>> = (0..n_coords).map(|_| zero_p()).collect();
2981        if coord_offset == 1 {
2982            let (ell_first, ell_second) = length_diag(&q_form);
2983            first[0] = ell_first;
2984            second_diag[0] = ell_second;
2985        }
2986        // Keep the Primary the builder would emit: the null component's shipped
2987        // matrix is a REBUILD off it, so the producer needs the same object to
2988        // differentiate the same thing (see below).
2989        single_scale_primary = Some(constructive_pullback_center_form(
2990            &geom.kz,
2991            &q_form,
2992            "measure-jet primary penalty",
2993        )?);
2994        if let (Some(primary), Some(frame)) = (
2995            single_scale_primary.as_mut(),
2996            measure_jet_primary_structural_null_frame(&geom.z, m, geom.head_lift.ncols())?,
2997        ) {
2998            *primary = primary.clone().with_structural_null_frame(
2999                frame,
3000                "measure-jet primary structural null declaration",
3001            )?;
3002        }
3003        vec![RawPenaltyJets {
3004            value: sandwich(&q_form),
3005            first,
3006            second_diag,
3007            cross: Vec::new(),
3008        }]
3009    };
3010
3011    if spec.double_penalty {
3012        let null_center =
3013            affine_function_nullspace_center_quadratic(geom.centers.view(), geom.masses.view())?;
3014        let null_form = null_center.dense();
3015        let mut first: Vec<Array2<f64>> = (0..n_coords).map(|_| zero_p()).collect();
3016        let mut second_diag: Vec<Array2<f64>> = (0..n_coords).map(|_| zero_p()).collect();
3017        if coord_offset == 1 {
3018            let (ell_first, ell_second) = length_diag(null_form);
3019            first[0] = ell_first;
3020            second_diag[0] = ell_second;
3021        }
3022        let mut value = sandwich(null_form);
3023        // The builder does NOT ship this raw pullback when a Primary exists: it
3024        // ships `rebuild_metric_consistent_ridge`'s output, `R = N M Nᵀ` with
3025        // `M = Nᵀ (EᵀH₀E) N` and `N` the Primary's declared structural null
3026        // frame. Differentiating the raw pullback instead is an
3027        // objective↔gradient desync on the `ln ℓ` coordinate, and #2761
3028        // measured it as the WHOLE of that coordinate's gradient error:
3029        //
3030        //   arm                       analytic     Ridders FD     rel
3031        //   double_penalty = true    -1.124278e1  -1.149171e1   2.2e-2
3032        //   double_penalty = false   -1.1255398e1 -1.1255398e1   9e-10
3033        //
3034        // with the total's per-atom breakdown putting it in `logdet_S`
3035        // (−0.2369) and `fixed_beta` (+0.4858). λ_null being tiny does not
3036        // shrink it: on the directions the Primary annihilates, `S_λ` IS
3037        // `λ_null·S_null`, so `λ_null` cancels out of
3038        // `tr(S_λ⁺ ∂S_λ/∂ψ)` and a wrong `∂S_null/∂ψ` lands at full size.
3039        //
3040        // The exact jets of the rebuilt object are `N (Nᵀ ∂S_raw N) Nᵀ`, since
3041        // `N` is ψ-fixed by construction (it is a declaration, not a rank
3042        // test). They are numerically ZERO here — `N`'s columns carry no
3043        // representer coefficients, so `E·N` is ℓ-invariant and `M` cannot
3044        // move — but computing them rather than asserting them keeps the
3045        // producer correct if a future frame does move.
3046        if let Some(primary) = single_scale_primary.as_ref() {
3047            let ridge_physical = ConstructiveQuadratic::from_energy_factor(
3048                null_center.factor().dot(&geom.kz),
3049                "measure-jet affine/null coefficient penalty",
3050            )?;
3051            match super::rebuild_metric_consistent_ridge(primary, &ridge_physical)? {
3052                Some(rebuilt) => {
3053                    let frame = primary
3054                        .structural_null_frame()
3055                        .cloned()
3056                        .unwrap_or_else(|| Array2::<f64>::zeros((p, 0)));
3057                    for coord in 0..n_coords {
3058                        first[coord] = restrict_jet_to_frame(&first[coord], &frame);
3059                        second_diag[coord] = restrict_jet_to_frame(&second_diag[coord], &frame);
3060                    }
3061                    value = rebuilt.dense().clone();
3062                }
3063                None => {
3064                    // The rebuild declined, so the builder ships an exact zero
3065                    // and the candidate is dropped. A dropped candidate has no
3066                    // derivative.
3067                    for coord in 0..n_coords {
3068                        first[coord] = zero_p();
3069                        second_diag[coord] = zero_p();
3070                    }
3071                    value = zero_p();
3072                }
3073            }
3074        }
3075        raw.push(RawPenaltyJets {
3076            value,
3077            first,
3078            second_diag,
3079            // H₀ is independent of α and τ; its only moving object is E(ℓ),
3080            // so every mixed coordinate derivative is zero.
3081            cross: (0..pairs.len()).map(|_| zero_p()).collect(),
3082        });
3083    }
3084
3085    let n_cands = raw.len();
3086    let mut penalties_first: Vec<Vec<Array2<f64>>> =
3087        (0..n_coords).map(|_| Vec::with_capacity(n_cands)).collect();
3088    let mut penalties_second_diag: Vec<Vec<Array2<f64>>> =
3089        (0..n_coords).map(|_| Vec::with_capacity(n_cands)).collect();
3090    // Cross matrices per pair per candidate, precomputed eagerly (the
3091    // candidate count is the band length, not the data size) and served
3092    // through the on-demand provider.
3093    let mut crosses: Vec<Vec<Array2<f64>>> = (0..pairs.len()).map(|_| Vec::new()).collect();
3094    for candidate in &raw {
3095        let s_raw = &candidate.value;
3096        // ONE Frobenius scale per candidate, fixed up front from `s_raw`
3097        // alone: c anchors the value and every derivative of this candidate.
3098        // `normalize_penaltywith_psi_derivatives` recomputes the identical c
3099        // per coordinate (same trace_of_product + sqrt on the same `s_raw`),
3100        // and its degenerate convention is mirrored here: ‖S‖_F ≤ 1e-12 (or
3101        // non-finite) reports scale 1.0 — the value passes through unscaled,
3102        // and the cross helper receives that same 1.0, never a collapsed
3103        // near-zero scale.
3104        let fro = trace_of_product(s_raw, s_raw).sqrt();
3105        let c = if fro.is_finite() && fro > 1e-12 {
3106            fro
3107        } else {
3108            1.0
3109        };
3110        for coord in 0..n_coords {
3111            let (_, s_first, s_second, _) = normalize_penaltywith_psi_derivatives(
3112                s_raw,
3113                &candidate.first[coord],
3114                &candidate.second_diag[coord],
3115            );
3116            penalties_first[coord].push(s_first);
3117            penalties_second_diag[coord].push(s_second);
3118        }
3119        for (pair_idx, &(a, b)) in pairs.iter().enumerate() {
3120            let cross_raw_mat = normalize_penalty_cross_psi_derivative(
3121                s_raw,
3122                &candidate.first[a],
3123                &candidate.first[b],
3124                &candidate.cross[pair_idx],
3125                c,
3126            );
3127            crosses[pair_idx].push(cross_raw_mat);
3128        }
3129    }
3130
3131    let pair_index: Vec<((usize, usize), Vec<Array2<f64>>)> =
3132        pairs.iter().copied().zip(crosses.into_iter()).collect();
3133    let provider = AnisoPenaltyCrossProvider::new(move |a, b| {
3134        pair_index
3135            .iter()
3136            .find(|((pa, pb), _)| (*pa, *pb) == (a, b) || (*pa, *pb) == (b, a))
3137            .map(|(_, mats)| mats.clone())
3138            .ok_or_else(|| {
3139                BasisError::InvalidInput(format!(
3140                    "measure-jet ψ cross derivative requested for unknown pair ({a}, {b})"
3141                ))
3142            })
3143    });
3144    let mut design_first: Vec<Array2<f64>> = (0..n_coords)
3145        .map(|_| Array2::<f64>::zeros((n, p)))
3146        .collect();
3147    let mut design_second_diag: Vec<Array2<f64>> = (0..n_coords)
3148        .map(|_| Array2::<f64>::zeros((n, p)))
3149        .collect();
3150    if let Some(jets) = &length_scale_jets {
3151        design_first[0] = jets.design_first.clone();
3152        design_second_diag[0] = jets.design_second.clone();
3153    }
3154    Ok(AnisoBasisPsiDerivatives {
3155        design_first,
3156        design_second_diag,
3157        design_second_cross: Vec::new(),
3158        design_second_cross_pairs: Vec::new(),
3159        penalties_first,
3160        penalties_second_diag,
3161        penalties_cross_pairs: pairs,
3162        penalties_cross_provider: Some(provider),
3163        implicit_operator: None,
3164    })
3165}
3166
3167#[cfg(test)]
3168mod tests {
3169    use super::*;
3170
3171    /// Deterministic Box–Muller standard normal from a 64-bit LCG state — a
3172    /// self-contained noise generator (no external RNG dependency).
3173    fn lcg_normal(state: &mut u64) -> f64 {
3174        let mut next = || {
3175            *state = state
3176                .wrapping_mul(6364136223846793005)
3177                .wrapping_add(1442695040888963407);
3178            // Top 53 bits → uniform (0, 1).
3179            (((*state >> 11) as f64) + 0.5) / (1u64 << 53) as f64
3180        };
3181        let u1 = next();
3182        let u2 = next();
3183        (-2.0 * u1.ln()).sqrt() * (std::f64::consts::TAU * u2).cos()
3184    }
3185
3186    /// The perpendicular off-manifold residual estimator recovers a KNOWN
3187    /// ambient noise scale on a 1-D manifold (a line) embedded in 2-D: points
3188    /// sampled along the tangent with isotropic-perpendicular Gaussian noise of
3189    /// scale σ, centers spaced along the line. The local-PCA smallest-eigenvalue
3190    /// floor must return ≈ σ (#2225).
3191    #[test]
3192    pub(crate) fn input_noise_scale_recovers_known_perpendicular_sigma() {
3193        // Line direction (unit) and its perpendicular in 2-D.
3194        let tang = [1.0 / 5f64.sqrt(), 2.0 / 5f64.sqrt()];
3195        let perp = [2.0 / 5f64.sqrt(), -1.0 / 5f64.sqrt()];
3196        let sigma = 0.05_f64;
3197        let n = 600usize;
3198        let mut state = 0x1234_5678_9abc_def0u64;
3199        let mut data = Array2::<f64>::zeros((n, 2));
3200        for j in 0..n {
3201            // Tangential coordinate marches deterministically over [0, 3].
3202            let t = 3.0 * (j as f64) / (n as f64 - 1.0);
3203            let noise = sigma * lcg_normal(&mut state);
3204            for a in 0..2 {
3205                data[(j, a)] = t * tang[a] + noise * perp[a];
3206            }
3207        }
3208        // Centers along the line (on the noiseless manifold): plenty of points
3209        // per cell to span the tangent.
3210        let n_centers = 8usize;
3211        let mut centers = Array2::<f64>::zeros((n_centers, 2));
3212        for i in 0..n_centers {
3213            let t = 3.0 * (i as f64 + 0.5) / (n_centers as f64);
3214            for a in 0..2 {
3215                centers[(i, a)] = t * tang[a];
3216            }
3217        }
3218        let est = measure_jet_input_noise_scale(data.view(), centers.view())
3219            .expect("estimate ok")
3220            .expect("noise scale present");
3221        // Sample smallest-eigenvalue floor is mildly downward-biased; require it
3222        // within 40% of the truth (central estimate, not a tuned tolerance).
3223        assert!(
3224            (est - sigma).abs() <= 0.4 * sigma,
3225            "estimated σ_coord {est} far from true {sigma}"
3226        );
3227    }
3228
3229    /// Too few points per cell (cannot span a d-dim tangent) ⇒ no estimate,
3230    /// so the caller leaves Var_input disabled rather than invent a scale.
3231    #[test]
3232    pub(crate) fn input_noise_scale_none_when_cells_too_small() {
3233        let data = array![[0.0, 0.0], [1.0, 2.0], [2.0, 4.0]];
3234        let centers = array![[0.0, 0.0], [1.0, 2.0], [2.0, 4.0]];
3235        // Each point is its own nearest center (1 point per cell < d + 1 = 3).
3236        assert!(
3237            measure_jet_input_noise_scale(data.view(), centers.view())
3238                .expect("estimate ok")
3239                .is_none()
3240        );
3241    }
3242
3243    pub(crate) fn two_cluster_centers() -> (ndarray::Array2<f64>, ndarray::Array1<f64>) {
3244        let centers = array![
3245            [0.00, 0.00],
3246            [0.31, 0.05],
3247            [0.58, -0.07],
3248            [0.93, 0.11],
3249            [1.22, 0.02],
3250            [1.49, -0.04],
3251            [3.10, 2.00],
3252            [3.42, 2.13],
3253            [3.71, 1.91],
3254            [4.05, 2.07],
3255            [4.33, 1.96],
3256            [4.61, 2.12],
3257        ];
3258        let m = centers.nrows();
3259        let masses = ndarray::Array1::<f64>::from_elem(m, 1.0 / m as f64);
3260        (centers, masses)
3261    }
3262    use ndarray::array;
3263
3264    pub(crate) fn band_for(centers: &Array2<f64>) -> MeasureJetBand {
3265        measure_jet_band(centers.view(), 0).expect("band")
3266    }
3267
3268    /// The no-mass contract: constants must be annihilated to machine
3269    /// precision at every scale (the constant is projected, never ridged).
3270    #[test]
3271    pub(crate) fn energy_form_annihilates_constants_exactly() {
3272        let (centers, masses) = two_cluster_centers();
3273        let band = band_for(&centers);
3274        let q = measure_jet_energy_form(centers.view(), masses.view(), &band, 1.5, 1.0, 1e-3)
3275            .expect("energy form");
3276        let m = q.nrows();
3277        let ones = Array1::<f64>::ones(m);
3278        let qv = q.dot(&ones);
3279        let scale = q.iter().fold(0.0_f64, |acc, v| acc.max(v.abs()));
3280        assert!(scale > 0.0, "energy form is identically zero");
3281        for (i, v) in qv.iter().enumerate() {
3282            assert!(
3283                v.abs() <= 1e-12 * scale,
3284                "Q·1 leak at row {i}: {v:.3e} vs scale {scale:.3e}"
3285            );
3286        }
3287        let vqv = ones.dot(&qv);
3288        assert!(
3289            vqv.abs() <= 1e-12 * scale,
3290            "constant carries energy: 1ᵀQ1 = {vqv:.3e}"
3291        );
3292    }
3293
3294    /// The default local projection annihilates ambient affine functions
3295    /// exactly; τ is retained for ψ layout but no longer adds an affine toll.
3296    #[test]
3297    pub(crate) fn energy_form_annihilates_affine_at_default_tau() {
3298        let (centers, masses) = two_cluster_centers();
3299        let band = band_for(&centers);
3300        let m = centers.nrows();
3301        // Affine values v = 0.7 + 1.3·x − 0.4·y, and a rough ±1 checkerboard.
3302        let mut affine = Array1::<f64>::zeros(m);
3303        let mut rough = Array1::<f64>::zeros(m);
3304        for i in 0..m {
3305            affine[i] = 0.7 + 1.3 * centers[(i, 0)] - 0.4 * centers[(i, 1)];
3306            rough[i] = if i % 2 == 0 { 1.0 } else { -1.0 };
3307        }
3308        let q = measure_jet_energy_form(centers.view(), masses.view(), &band, 1.5, 1.0, 1e-3)
3309            .expect("energy form");
3310        let e_affine = affine.dot(&q.dot(&affine));
3311        let e_rough = rough.dot(&q.dot(&rough));
3312        assert!(e_rough > 0.0, "rough vector must pay energy");
3313        assert!(
3314            e_affine.abs() <= 1e-12 * e_rough,
3315            "default affine energy {e_affine:.3e} vs rough {e_rough:.3e}"
3316        );
3317    }
3318
3319    /// PSD: the energy is a sum of weighted least-squares residuals.
3320    #[test]
3321    pub(crate) fn energy_form_is_psd() {
3322        let (centers, masses) = two_cluster_centers();
3323        let band = band_for(&centers);
3324        let q = measure_jet_energy_form(centers.view(), masses.view(), &band, 1.5, 1.0, 1e-3)
3325            .expect("energy form");
3326        let m = q.nrows();
3327        for trial in 0..5usize {
3328            let v = Array1::<f64>::from_shape_fn(m, |i| {
3329                ((i * 7 + trial * 13) % 11) as f64 / 11.0 - 0.5
3330            });
3331            let e = v.dot(&q.dot(&v));
3332            assert!(e >= -1e-10, "vᵀQv = {e:.3e} < 0 on trial {trial}");
3333        }
3334    }
3335
3336    /// A 1-D filament embedded in 2-D: high-frequency center values along the
3337    /// strand pay strictly more energy than a slow trend.
3338    #[test]
3339    pub(crate) fn rough_vector_pays_more_than_smooth() {
3340        let m = 24usize;
3341        let centers = Array2::<f64>::from_shape_fn((m, 2), |(i, k)| {
3342            let t = i as f64 / (m as f64 - 1.0);
3343            if k == 0 {
3344                t * 4.0
3345            } else {
3346                0.3 * (t * 4.0).sin()
3347            }
3348        });
3349        let masses = Array1::<f64>::from_elem(m, 1.0 / m as f64);
3350        let band = band_for(&centers);
3351        let q = measure_jet_energy_form(centers.view(), masses.view(), &band, 1.5, 1.0, 1e-3)
3352            .expect("energy form");
3353        let slow = Array1::<f64>::from_shape_fn(m, |i| (i as f64 / (m as f64 - 1.0)).powi(2));
3354        let fast = Array1::<f64>::from_shape_fn(m, |i| if i % 2 == 0 { 0.5 } else { -0.5 });
3355        let e_slow = slow.dot(&q.dot(&slow));
3356        let e_fast = fast.dot(&q.dot(&fast));
3357        assert!(
3358            e_fast > 10.0 * e_slow,
3359            "alternating values must pay >> a slow trend: fast {e_fast:.3e} vs slow {e_slow:.3e}"
3360        );
3361    }
3362
3363    /// The exact (s, α) jets and zero τ slots must match central finite
3364    /// differences of the energy — the FD gate the ψ-channel stage will
3365    /// inherit (the discipline whose absence is exactly the
3366    /// objective↔gradient desync bug class).
3367    #[test]
3368    pub(crate) fn energy_jets_match_finite_differences() {
3369        let (centers, masses) = two_cluster_centers();
3370        let band = band_for(&centers);
3371        let (s0, a0, tau) = (1.3, 0.8, 1e-3);
3372        let jets =
3373            measure_jet_energy_form_with_jets(centers.view(), masses.view(), &band, s0, a0, tau)
3374                .expect("jets");
3375        let q_at = |s: f64, a: f64| {
3376            measure_jet_energy_form(centers.view(), masses.view(), &band, s, a, tau)
3377                .expect("energy form")
3378        };
3379        // Base form must equal the plain assembly bit-for-bit (same walk).
3380        let q_plain = q_at(s0, a0);
3381        for (a, b) in jets.q.iter().zip(q_plain.iter()) {
3382            assert!(
3383                (a - b).abs() <= 1e-14 * (1.0 + b.abs()),
3384                "Q drift {a} vs {b}"
3385            );
3386        }
3387        let lt0 = tau.ln();
3388        let q_at_lt = |lt: f64| {
3389            measure_jet_energy_form(centers.view(), masses.view(), &band, s0, a0, lt.exp())
3390                .expect("energy form")
3391        };
3392        // FD step calibrated for the SECOND differences: their roundoff
3393        // floor is ~4·ε_f64·scale/h² (assembly noise amplified by 1/h²), so
3394        // h = 1e-4 ≈ ε^(1/4) balances it against the O(h²) truncation —
3395        // both land ≥3 orders below the unchanged 5e-5·scale gate. h = 1e-5
3396        // sits ON the roundoff floor and fails spuriously.
3397        let h = 1e-4;
3398        let checks: [(&str, &Array2<f64>, Array2<f64>); 9] = [
3399            ("dq_ds", &jets.dq_ds, {
3400                let (p, m_) = (q_at(s0 + h, a0), q_at(s0 - h, a0));
3401                (&p - &m_) / (2.0 * h)
3402            }),
3403            ("d2q_ds2", &jets.d2q_ds2, {
3404                let (p, c, m_) = (q_at(s0 + h, a0), q_at(s0, a0), q_at(s0 - h, a0));
3405                (&(&p + &m_) - &(&c * 2.0)) / (h * h)
3406            }),
3407            ("dq_dalpha", &jets.dq_dalpha, {
3408                let (p, m_) = (q_at(s0, a0 + h), q_at(s0, a0 - h));
3409                (&p - &m_) / (2.0 * h)
3410            }),
3411            ("d2q_dalpha2", &jets.d2q_dalpha2, {
3412                let (p, c, m_) = (q_at(s0, a0 + h), q_at(s0, a0), q_at(s0, a0 - h));
3413                (&(&p + &m_) - &(&c * 2.0)) / (h * h)
3414            }),
3415            ("d2q_ds_dalpha", &jets.d2q_ds_dalpha, {
3416                let pp = q_at(s0 + h, a0 + h);
3417                let pm = q_at(s0 + h, a0 - h);
3418                let mp = q_at(s0 - h, a0 + h);
3419                let mm = q_at(s0 - h, a0 - h);
3420                (&(&pp - &pm) - &(&mp - &mm)) / (4.0 * h * h)
3421            }),
3422            ("dq_dlogtau", &jets.dq_dlogtau, {
3423                let (p, m_) = (q_at_lt(lt0 + h), q_at_lt(lt0 - h));
3424                (&p - &m_) / (2.0 * h)
3425            }),
3426            ("d2q_dlogtau2", &jets.d2q_dlogtau2, {
3427                let (p, c, m_) = (q_at_lt(lt0 + h), q_at_lt(lt0), q_at_lt(lt0 - h));
3428                (&(&p + &m_) - &(&c * 2.0)) / (h * h)
3429            }),
3430            ("d2q_ds_dlogtau", &jets.d2q_ds_dlogtau, {
3431                let f = |s: f64, lt: f64| {
3432                    measure_jet_energy_form(centers.view(), masses.view(), &band, s, a0, lt.exp())
3433                        .expect("energy form")
3434                };
3435                let pp = f(s0 + h, lt0 + h);
3436                let pm = f(s0 + h, lt0 - h);
3437                let mp = f(s0 - h, lt0 + h);
3438                let mm = f(s0 - h, lt0 - h);
3439                (&(&pp - &pm) - &(&mp - &mm)) / (4.0 * h * h)
3440            }),
3441            ("d2q_dalpha_dlogtau", &jets.d2q_dalpha_dlogtau, {
3442                let f = |a: f64, lt: f64| {
3443                    measure_jet_energy_form(centers.view(), masses.view(), &band, s0, a, lt.exp())
3444                        .expect("energy form")
3445                };
3446                let pp = f(a0 + h, lt0 + h);
3447                let pm = f(a0 + h, lt0 - h);
3448                let mp = f(a0 - h, lt0 + h);
3449                let mm = f(a0 - h, lt0 - h);
3450                (&(&pp - &pm) - &(&mp - &mm)) / (4.0 * h * h)
3451            }),
3452        ];
3453        for (name, analytic, fd) in checks.iter() {
3454            let scale = fd.iter().fold(1e-30_f64, |acc, v| acc.max(v.abs()));
3455            for (a, b) in analytic.iter().zip(fd.iter()) {
3456                assert!(
3457                    (a - b).abs() <= 5e-5 * scale,
3458                    "{name} jet mismatch: analytic {a:.6e} vs FD {b:.6e} (scale {scale:.3e})"
3459                );
3460            }
3461        }
3462    }
3463
3464    /// The per-scale spectrum must sum exactly to the total energy (same
3465    /// blocks, one-hot weights) and concentrate rough content at fine
3466    /// scales.
3467    #[test]
3468    pub(crate) fn scale_spectrum_sums_to_total_and_localizes_roughness() {
3469        let m = 24usize;
3470        let centers = Array2::<f64>::from_shape_fn((m, 2), |(i, k)| {
3471            let t = i as f64 / (m as f64 - 1.0);
3472            if k == 0 { t * 4.0 } else { 0.0 }
3473        });
3474        let masses = Array1::<f64>::from_elem(m, 1.0 / m as f64);
3475        let band = band_for(&centers);
3476        let q = measure_jet_energy_form(centers.view(), masses.view(), &band, 1.5, 1.0, 1e-3)
3477            .expect("energy form");
3478        let fast = Array1::<f64>::from_shape_fn(m, |i| if i % 2 == 0 { 0.5 } else { -0.5 });
3479        let spec = measure_jet_scale_spectrum(
3480            centers.view(),
3481            masses.view(),
3482            &band,
3483            1.5,
3484            1.0,
3485            1e-3,
3486            fast.view(),
3487        )
3488        .expect("spectrum");
3489        assert_eq!(spec.len(), band.eps.len());
3490        let total = fast.dot(&q.dot(&fast));
3491        let sum: f64 = spec.iter().sum();
3492        assert!(
3493            (sum - total).abs() <= 1e-10 * total.abs().max(1e-30),
3494            "spectrum must sum to vᵀQv: {sum:.6e} vs {total:.6e}"
3495        );
3496        // Alternating-sign content lives at the finest scale of the band.
3497        let finest = spec[0];
3498        let coarsest = *spec.last().expect("nonempty spectrum");
3499        assert!(
3500            finest > coarsest,
3501            "alternating values must charge fine scales hardest: fine {finest:.3e} vs coarse {coarsest:.3e}"
3502        );
3503    }
3504
3505    /// The support curve separates on-web from off-web queries at fine
3506    /// scales and grows monotonically in ε for any query.
3507    #[test]
3508    pub(crate) fn support_curve_separates_on_web_from_off_web() {
3509        let m = 24usize;
3510        let centers = Array2::<f64>::from_shape_fn((m, 2), |(i, k)| {
3511            let t = i as f64 / (m as f64 - 1.0);
3512            if k == 0 { t * 4.0 } else { 0.0 }
3513        });
3514        let masses = Array1::<f64>::from_elem(m, 1.0 / m as f64);
3515        let band = band_for(&centers);
3516        let queries = array![[2.0, 0.0], [2.0, 1.5]];
3517        let curves =
3518            measure_jet_support_curve(queries.view(), centers.view(), masses.view(), &band.eps)
3519                .expect("support curve");
3520        // On-web sees strictly more mass than off-web at the finest scale.
3521        assert!(
3522            curves[(0, 0)] > 10.0 * curves[(1, 0)],
3523            "fine-scale support must separate web from void: on {:.3e} vs off {:.3e}",
3524            curves[(0, 0)],
3525            curves[(1, 0)]
3526        );
3527        // Kernel mass is monotone in ε for every query.
3528        for qi in 0..2 {
3529            for li in 1..band.eps.len() {
3530                assert!(
3531                    curves[(qi, li)] >= curves[(qi, li - 1)] - 1e-15,
3532                    "support curve must be monotone in scale (query {qi}, level {li})"
3533                );
3534            }
3535        }
3536    }
3537
3538    /// The default is single-scale mode at ANY center count: one Primary
3539    /// jet-energy candidate plus the independently REML-selected affine/null
3540    /// component requested by the default `double_penalty`. Multiscale (the
3541    /// per-scale spectral split + ψ dials) is an EXPLICIT opt-in
3542    /// (`spec.multiscale`, the DSL `mjs(…, multiscale=true)`) — there is no
3543    /// center-count auto-gate (#1116). `measure_jet_multiscale_mode` is the
3544    /// single source for this decision.
3545    #[test]
3546    pub(crate) fn default_stays_single_scale_until_multiscale_opt_in() {
3547        let n = 200usize;
3548        let data = Array2::<f64>::from_shape_fn((n, 2), |(i, k)| {
3549            let t = i as f64 / (n as f64 - 1.0);
3550            if k == 0 {
3551                t * 3.0
3552            } else {
3553                0.4 * (t * 3.0).sin()
3554            }
3555        });
3556        // Default (multiscale = false) stays single-scale even at a LARGE center
3557        // count that, under the deleted auto-gate, would have flipped to
3558        // multiscale: one pure Primary plus one function-space null component.
3559        let single = MeasureJetBasisSpec {
3560            center_strategy: CenterStrategy::FarthestPoint { num_centers: 80 },
3561            ..MeasureJetBasisSpec::default()
3562        };
3563        assert!(
3564            !measure_jet_multiscale_mode(&single),
3565            "default must resolve to single-scale at any center count"
3566        );
3567        let built_single =
3568            build_measure_jet_basis(data.view(), &single).expect("single-scale build");
3569        assert_eq!(
3570            built_single.active_penalties.len(),
3571            2,
3572            "single-scale double-penalty mode emits Primary + affine/null component"
3573        );
3574        assert!(matches!(
3575            built_single.active_penalties[0].info.source,
3576            PenaltySource::Primary
3577        ));
3578        assert!(matches!(
3579            built_single.active_penalties[1].info.source,
3580            PenaltySource::DoublePenaltyNullspace
3581        ));
3582        // The explicit opt-in flips to multiscale at the SAME center count: the
3583        // per-scale spectral split (several candidates) plus the same explicit
3584        // null-component candidate, strictly more candidates than single-scale.
3585        let multi = MeasureJetBasisSpec {
3586            center_strategy: CenterStrategy::FarthestPoint { num_centers: 80 },
3587            multiscale: true,
3588            ..MeasureJetBasisSpec::default()
3589        };
3590        assert!(
3591            measure_jet_multiscale_mode(&multi),
3592            "multiscale=true must resolve to multiscale mode"
3593        );
3594        let built_multi = build_measure_jet_basis(data.view(), &multi).expect("multiscale build");
3595        assert!(
3596            built_multi.active_penalties.len() > built_single.active_penalties.len(),
3597            "multiscale mode emits the per-scale spectral split plus null selection, got {} (vs single-scale {})",
3598            built_multi.active_penalties.len(),
3599            built_single.active_penalties.len()
3600        );
3601    }
3602
3603    /// An explicit order pins the Mellin weights and fuses the band into a
3604    /// single Primary candidate. Disabling explicit null recovery leaves exactly
3605    /// that candidate; enabling it must never alter the Primary itself.
3606    #[test]
3607    pub(crate) fn fused_mode_without_double_penalty_emits_single_primary_candidate() {
3608        let n = 40usize;
3609        let data = Array2::<f64>::from_shape_fn((n, 2), |(i, k)| {
3610            let t = i as f64 / (n as f64 - 1.0);
3611            if k == 0 {
3612                t * 3.0
3613            } else {
3614                0.4 * (t * 3.0).sin()
3615            }
3616        });
3617        let spec = MeasureJetBasisSpec {
3618            center_strategy: CenterStrategy::FarthestPoint { num_centers: 14 },
3619            order_s: 1.3,
3620            double_penalty: false,
3621            ..MeasureJetBasisSpec::default()
3622        };
3623        let built = build_measure_jet_basis(data.view(), &spec).expect("fused build");
3624        assert_eq!(
3625            built.active_penalties.len(),
3626            1,
3627            "single-scale mode without null recovery emits exactly one Primary"
3628        );
3629        assert!(matches!(
3630            built.active_penalties[0].info.source,
3631            PenaltySource::Primary
3632        ));
3633        let BasisMetadata::MeasureJet { order_s, .. } = &built.metadata else {
3634            panic!("measure-jet build must return MeasureJet metadata");
3635        };
3636        assert_eq!(*order_s, 1.3, "explicit order must persist verbatim");
3637    }
3638
3639    /// The single-scale affine head is a gauge-fixed decomposition, not a
3640    /// coefficient ridge: RBF center values are exactly mass-orthogonal to the
3641    /// supported affine space, and replacing those directions with the head
3642    /// keeps the RAW chart exactly `m` wide. The collection's parametric
3643    /// orthogonalization then removes the head's constant, landing the FIT
3644    /// chart at `m - 1` — the width this test asserted directly before #2751,
3645    /// when the head omitted the constant and the centering took a linear
3646    /// direction instead.
3647    #[test]
3648    pub(crate) fn single_scale_affine_head_gauge_annihilates_center_cross() {
3649        let n = 90usize;
3650        let data = Array2::<f64>::from_shape_fn((n, 2), |(i, k)| {
3651            let t = i as f64 / (n as f64 - 1.0);
3652            if k == 0 {
3653                3.0 * t
3654            } else {
3655                (2.0 * std::f64::consts::PI * t).sin() + 0.2 * t
3656            }
3657        });
3658        let spec = MeasureJetBasisSpec {
3659            center_strategy: CenterStrategy::FarthestPoint { num_centers: 18 },
3660            double_penalty: false,
3661            multiscale: false,
3662            ..MeasureJetBasisSpec::default()
3663        };
3664        let geom = realize_measure_jet_geometry(data.view(), &spec).expect("realized geometry");
3665        let m = geom.centers.nrows();
3666        let head_width = geom.head_lift.ncols();
3667        assert!(head_width > 0, "fixture must realize an affine head");
3668        assert_eq!(
3669            geom.z.ncols(),
3670            m,
3671            "the affine head replaces the RBF block's affine directions one for one: the RAW \
3672             chart is exactly m wide (m - head_width representers + head_width head columns). \
3673             The collection's parametric orthogonalization then removes the constant, landing \
3674             the FIT chart at m - 1 (#2751)"
3675        );
3676        let rbf_rank = m - head_width;
3677        let z_rbf = geom.z.slice(ndarray::s![..m, ..rbf_rank]).to_owned();
3678        let k_cc =
3679            measure_jet_design_matrix(geom.centers.view(), geom.centers.view(), geom.length_scale)
3680                .expect("center kernel");
3681        let affine = measure_jet_affine_value_basis(geom.centers.view(), geom.masses.view());
3682        assert_eq!(affine.ncols(), head_width);
3683        let mut weighted_affine = affine.clone();
3684        for (i, mut row) in weighted_affine.outer_iter_mut().enumerate() {
3685            row.mapv_inplace(|v| v * geom.masses[i]);
3686        }
3687        let constraint_cross = k_cc.t().dot(&weighted_affine);
3688        let residual = constraint_cross.t().dot(&z_rbf);
3689        let scale = constraint_cross
3690            .iter()
3691            .fold(1.0_f64, |acc, value| acc.max(value.abs()));
3692        assert!(
3693            residual.iter().all(|value| value.abs() <= 1e-10 * scale),
3694            "A^T W Kcc Z_rbf must vanish; max residual {:.3e}",
3695            residual
3696                .iter()
3697                .fold(0.0_f64, |acc, value| acc.max(value.abs()))
3698        );
3699    }
3700
3701    /// A function-space penalty must transform covariantly with its evaluation
3702    /// map. This directly excludes any hidden Euclidean coefficient projector.
3703    #[test]
3704    pub(crate) fn affine_null_penalty_is_covariant_under_coefficient_reparameterization() {
3705        let centers = array![
3706            [-1.0, 0.2],
3707            [-0.4, -0.3],
3708            [0.1, 0.5],
3709            [0.7, -0.2],
3710            [1.2, 0.4],
3711            [1.8, -0.1],
3712        ];
3713        let masses = array![0.08, 0.12, 0.18, 0.22, 0.17, 0.23];
3714        let evaluation = Array2::<f64>::from_shape_fn((centers.nrows(), 3), |(i, j)| {
3715            ((i + 2 * j + 1) as f64).sin() + 0.15 * (i * (j + 1)) as f64
3716        });
3717        let reparameterization = array![[1.7, 0.2, -0.1], [0.0, 0.6, 0.3], [0.0, 0.0, 1.3]];
3718        let base = affine_function_nullspace_quadratic(&evaluation, centers.view(), masses.view())
3719            .expect("base function-space penalty")
3720            .into_dense();
3721        let transformed_evaluation = evaluation.dot(&reparameterization);
3722        let transformed = affine_function_nullspace_quadratic(
3723            &transformed_evaluation,
3724            centers.view(),
3725            masses.view(),
3726        )
3727        .expect("reparameterized function-space penalty")
3728        .into_dense();
3729        let expected = reparameterization.t().dot(&base).dot(&reparameterization);
3730        let scale = expected
3731            .iter()
3732            .fold(1.0_f64, |acc, value| acc.max(value.abs()));
3733        assert!(
3734            transformed
3735                .iter()
3736                .zip(expected.iter())
3737                .all(|(actual, want)| (actual - want).abs() <= 1e-11 * scale),
3738            "S(E R) must equal R^T S(E) R"
3739        );
3740    }
3741
3742    /// `double_penalty` adds a distinct evidence-selected component and cannot
3743    /// mutate the jet-energy estimand carried by Primary.
3744    #[test]
3745    pub(crate) fn double_penalty_leaves_primary_matrix_unchanged() {
3746        let n = 64usize;
3747        let data = Array2::<f64>::from_shape_fn((n, 2), |(i, k)| {
3748            let t = i as f64 / (n as f64 - 1.0);
3749            if k == 0 { 2.5 * t } else { (4.0 * t).cos() }
3750        });
3751        let base = MeasureJetBasisSpec {
3752            center_strategy: CenterStrategy::FarthestPoint { num_centers: 16 },
3753            order_s: 1.25,
3754            double_penalty: false,
3755            ..MeasureJetBasisSpec::default()
3756        };
3757        let without = build_measure_jet_basis(data.view(), &base).expect("primary-only build");
3758        let with = build_measure_jet_basis(
3759            data.view(),
3760            &MeasureJetBasisSpec {
3761                double_penalty: true,
3762                ..base.clone()
3763            },
3764        )
3765        .expect("double-penalty build");
3766        assert_eq!(without.active_penalties.len(), 1);
3767        assert_eq!(with.active_penalties.len(), 2);
3768        assert!(matches!(
3769            without.active_penalties[0].info.source,
3770            PenaltySource::Primary
3771        ));
3772        assert!(matches!(
3773            with.active_penalties[0].info.source,
3774            PenaltySource::Primary
3775        ));
3776        assert!(matches!(
3777            with.active_penalties[1].info.source,
3778            PenaltySource::DoublePenaltyNullspace
3779        ));
3780        assert!(
3781            without.active_penalties[0]
3782                .matrix
3783                .iter()
3784                .zip(with.active_penalties[0].matrix.iter())
3785                .all(|(a, b)| (a - b).abs() <= 1e-13),
3786            "turning on null recovery must not modify Primary"
3787        );
3788    }
3789
3790    /// The Householder basis must be orthonormal with sum-to-zero columns.
3791    #[test]
3792    pub(crate) fn householder_sum_to_zero_basis_is_orthonormal() {
3793        let m = 9usize;
3794        let u = householder_sum_to_zero_u(m);
3795        let z = householder_sum_to_zero_z(&u);
3796        for j in 0..(m - 1) {
3797            let col_j = z.column(j);
3798            assert!(col_j.sum().abs() <= 1e-12, "column {j} must sum to zero");
3799            for j2 in j..(m - 1) {
3800                let dot = col_j.dot(&z.column(j2));
3801                let want = if j == j2 { 1.0 } else { 0.0 };
3802                assert!(
3803                    (dot - want).abs() <= 1e-12,
3804                    "orthonormality failure at ({j}, {j2}): {dot}"
3805                );
3806            }
3807        }
3808    }
3809
3810    /// Frozen-geometry fixture shared by the ψ-producer FD gates: build
3811    /// once, pin everything (nodes, masses, band, transform, realized ℓ),
3812    /// and return the pinned spec so dial-perturbed rebuilds move ONLY the
3813    /// dials — the per-trial contract the optimizer relies on.
3814    pub(crate) fn frozen_spec_fixture(
3815        order_s: f64,
3816        multiscale: bool,
3817    ) -> (Array2<f64>, MeasureJetBasisSpec) {
3818        // Multiscale (per-scale + ψ) mode is the explicit opt-in (#1116); the
3819        // per-level fixture passes `multiscale = true`, the fused fixture
3820        // `false`. A large center count is kept so the multiscale spectrum is
3821        // identifiable when opted in.
3822        let n = 140usize;
3823        let data = Array2::<f64>::from_shape_fn((n, 2), |(i, k)| {
3824            let t = i as f64 / (n as f64 - 1.0);
3825            if k == 0 {
3826                t * 3.0
3827            } else {
3828                0.5 * (t * 3.0).cos() + if i % 9 == 0 { 0.8 } else { 0.0 }
3829            }
3830        });
3831        let spec = MeasureJetBasisSpec {
3832            center_strategy: CenterStrategy::FarthestPoint { num_centers: 70 },
3833            order_s,
3834            multiscale,
3835            // These fixtures gate the PENALTY-dial derivatives; freeze ℓ so the
3836            // coordinate layout is exactly the penalty dials (the design-moving
3837            // ℓ dial has its own FD gate, `psi_producer_matches_fd_length_scale`).
3838            learn_length_scale: false,
3839            ..MeasureJetBasisSpec::default()
3840        };
3841        let first = build_measure_jet_basis(data.view(), &spec).expect("fixture build");
3842        let BasisMetadata::MeasureJet {
3843            centers,
3844            length_scale,
3845            eps_band,
3846            masses,
3847            support_means,
3848            penalty_normalization_scales,
3849            raw_penalty_normalization_scales,
3850            fused_penalty_normalization_scale,
3851            constraint_transform,
3852            ..
3853        } = &first.metadata
3854        else {
3855            panic!("measure-jet build must return MeasureJet metadata");
3856        };
3857        let frozen = MeasureJetBasisSpec {
3858            center_strategy: CenterStrategy::UserProvided(centers.clone()),
3859            order_s,
3860            alpha: spec.alpha,
3861            tau0: spec.tau0,
3862            num_scales: eps_band.len(),
3863            // MeasureJet freezes its range STANDARDIZED and replays it
3864            // verbatim; the tag is what records that it is the odd family out.
3865            length_scale: length_scale.standardized_value(),
3866            double_penalty: spec.double_penalty,
3867            learn_length_scale: false,
3868            multiscale,
3869            identifiability: MeasureJetIdentifiability::FrozenTransform {
3870                transform: constraint_transform.clone().expect("fit-time z"),
3871            },
3872            frozen_quadrature: Some(MeasureJetFrozenQuadrature {
3873                masses: masses.clone(),
3874                eps_band: eps_band.clone(),
3875                support_means: support_means.clone(),
3876                penalty_normalization_scales: penalty_normalization_scales.clone(),
3877                raw_penalty_normalization_scales: raw_penalty_normalization_scales.clone(),
3878                fused_penalty_normalization_scale: *fused_penalty_normalization_scale,
3879                sigma_coord: None,
3880            }),
3881        };
3882        (data, frozen)
3883    }
3884
3885    /// ψ-producer vs central finite differences of the NORMALIZED fit-time
3886    /// candidates under frozen geometry — per-level mode (coords α, lnτ).
3887    /// This is the end-to-end gate #901 never had: the derivative is checked
3888    /// against the exact object the optimizer consumes.
3889    #[test]
3890    pub(crate) fn psi_producer_matches_fd_per_level_mode() {
3891        let (data, frozen) = frozen_spec_fixture(0.0, true);
3892        let derivs =
3893            build_measure_jet_basis_psi_derivatives(data.view(), &frozen).expect("psi derivatives");
3894        let l_count = frozen
3895            .frozen_quadrature
3896            .as_ref()
3897            .expect("frozen quadrature")
3898            .eps_band
3899            .len();
3900        assert_eq!(
3901            derivs.penalties_first.len(),
3902            2,
3903            "per-level coords are (α, lnτ)"
3904        );
3905        assert_eq!(derivs.penalties_first[0].len(), l_count + 1);
3906        assert_eq!(derivs.penalties_cross_pairs, vec![(0, 1)]);
3907        let pen_at = |alpha: f64, tau0: f64| {
3908            let trial = MeasureJetBasisSpec {
3909                alpha,
3910                tau0,
3911                ..frozen.clone()
3912            };
3913            build_measure_jet_basis(data.view(), &trial)
3914                .expect("trial build")
3915                .active_penalties
3916                .into_iter()
3917                .map(|penalty| penalty.matrix)
3918                .collect::<Vec<_>>()
3919        };
3920        // Second-difference-optimal step (see the jets FD test): the 4-point
3921        // cross stencil shares the ~ε·scale/h² roundoff floor.
3922        let h = 1e-4;
3923        let (a0, t0) = (frozen.alpha, frozen.tau0);
3924        let ap = pen_at(a0 + h, t0);
3925        let am = pen_at(a0 - h, t0);
3926        let tp = pen_at(a0, t0 * h.exp());
3927        let tm = pen_at(a0, t0 * (-h).exp());
3928        assert_eq!(
3929            ap.len(),
3930            l_count + 1,
3931            "fixture must keep every scale active"
3932        );
3933        for level in 0..l_count {
3934            let fd_alpha = (&ap[level] - &am[level]) / (2.0 * h);
3935            let fd_tau = (&tp[level] - &tm[level]) / (2.0 * h);
3936            for (name, analytic, fd) in [
3937                ("alpha", &derivs.penalties_first[0][level], fd_alpha),
3938                ("ln_tau", &derivs.penalties_first[1][level], fd_tau),
3939            ] {
3940                let scale = fd.iter().fold(1e-30_f64, |acc, v| acc.max(v.abs()));
3941                for (x, y) in analytic.iter().zip(fd.iter()) {
3942                    assert!(
3943                        (x - y).abs() <= 5e-5 * scale,
3944                        "{name} jet of scale-candidate {level}: analytic {x:.6e} vs FD {y:.6e}"
3945                    );
3946                }
3947            }
3948        }
3949        // The function-space null candidate is independent of α and τ.
3950        for coord in 0..2 {
3951            assert!(
3952                derivs.penalties_first[coord][l_count]
3953                    .iter()
3954                    .all(|v| *v == 0.0),
3955                "null-component candidate must have zero (α, lnτ) drift"
3956            );
3957        }
3958        // Cross derivative through the provider, against a 4-point FD.
3959        let provider = derivs
3960            .penalties_cross_provider
3961            .as_ref()
3962            .expect("cross provider");
3963        let cross = provider.evaluate(0, 1).expect("cross pair (α, lnτ)");
3964        let pp = pen_at(a0 + h, t0 * h.exp());
3965        let pm = pen_at(a0 + h, t0 * (-h).exp());
3966        let mp = pen_at(a0 - h, t0 * h.exp());
3967        let mm = pen_at(a0 - h, t0 * (-h).exp());
3968        for level in 0..l_count {
3969            let fd = (&(&pp[level] - &pm[level]) - &(&mp[level] - &mm[level])) / (4.0 * h * h);
3970            let scale = fd.iter().fold(1e-30_f64, |acc, v| acc.max(v.abs()));
3971            for (x, y) in cross[level].iter().zip(fd.iter()) {
3972                assert!(
3973                    (x - y).abs() <= 5e-4 * scale,
3974                    "cross (α, lnτ) jet of scale-candidate {level}: analytic {x:.6e} vs FD {y:.6e}"
3975                );
3976            }
3977        }
3978    }
3979
3980    /// Design-moving ℓ dial (#1116): the producer's design jets and every
3981    /// normalized penalty candidate's jets must match central differences of the
3982    /// REBUILT objects under frozen geometry. Although the center-value forms
3983    /// `Q` and `H₀` are ℓ-invariant, their coefficient pullbacks `E(ℓ)ᵀQ E(ℓ)`
3984    /// and `E(ℓ)ᵀH₀E(ℓ)` are not.
3985    #[test]
3986    pub(crate) fn psi_producer_matches_fd_length_scale() {
3987        // Single-scale with opt-in ℓ learning; frozen geometry so only ℓ moves
3988        // across the FD trials.
3989        let (data, mut frozen) = frozen_spec_fixture(0.0, false);
3990        frozen.learn_length_scale = true;
3991        let derivs =
3992            build_measure_jet_basis_psi_derivatives(data.view(), &frozen).expect("psi derivatives");
3993        // ℓ is the only coordinate in single-scale + learn_length_scale.
3994        assert_eq!(
3995            derivs.design_first.len(),
3996            1,
3997            "single-scale + learn_length_scale enrolls exactly the ℓ coordinate"
3998        );
3999        assert_eq!(
4000            derivs.penalties_first[0].len(),
4001            2,
4002            "single-scale double penalty carries Primary + affine/null component"
4003        );
4004        // Rebuild design and normalized penalties at ℓ·e^{±h}; the explicit
4005        // positive length_scale is honored verbatim while the frozen transform
4006        // keeps the coefficient chart fixed.
4007        let ell0 = frozen.length_scale;
4008        let build_at = |ell: f64| {
4009            let trial = MeasureJetBasisSpec {
4010                length_scale: ell,
4011                ..frozen.clone()
4012            };
4013            build_measure_jet_basis(data.view(), &trial).expect("trial build")
4014        };
4015        let h: f64 = 1e-4;
4016        let plus = build_at(ell0 * h.exp());
4017        let minus = build_at(ell0 * (-h).exp());
4018        let at = build_at(ell0);
4019        assert_eq!(
4020            plus.active_penalties.len(),
4021            2,
4022            "fixture must keep both candidates active"
4023        );
4024        assert_eq!(
4025            minus.active_penalties.len(),
4026            2,
4027            "fixture must keep both candidates active"
4028        );
4029        assert_eq!(
4030            at.active_penalties.len(),
4031            2,
4032            "fixture must keep both candidates active"
4033        );
4034
4035        let x_plus = plus.design.to_dense();
4036        let x_minus = minus.design.to_dense();
4037        let x_0 = at.design.to_dense();
4038        let fd_first = (&x_plus - &x_minus) / (2.0 * h);
4039        let fd_second = (&x_plus - &(&x_0 * 2.0) + &x_minus) / (h * h);
4040        let scale1 = fd_first.iter().fold(1e-30_f64, |acc, v| acc.max(v.abs()));
4041        for (x, y) in derivs.design_first[0].iter().zip(fd_first.iter()) {
4042            assert!(
4043                (x - y).abs() <= 5e-5 * scale1,
4044                "∂X/∂lnℓ: analytic {x:.6e} vs FD {y:.6e}"
4045            );
4046        }
4047        let scale2 = fd_second.iter().fold(1e-30_f64, |acc, v| acc.max(v.abs()));
4048        for (x, y) in derivs.design_second_diag[0].iter().zip(fd_second.iter()) {
4049            assert!(
4050                (x - y).abs() <= 1e-3 * scale2,
4051                "∂²X/∂lnℓ²: analytic {x:.6e} vs FD {y:.6e}"
4052            );
4053        }
4054
4055        for candidate in 0..2 {
4056            let fd_penalty_first = (&plus.active_penalties[candidate].matrix
4057                - &minus.active_penalties[candidate].matrix)
4058                / (2.0 * h);
4059            let fd_penalty_second = (&plus.active_penalties[candidate].matrix
4060                - &(&at.active_penalties[candidate].matrix * 2.0)
4061                + &minus.active_penalties[candidate].matrix)
4062                / (h * h);
4063            // A central difference cannot resolve a derivative below its own
4064            // cancellation noise: differencing entries of size `E` at step `h`
4065            // leaves `~ε·E/h` in the first difference and `~ε·E/h²` in the
4066            // second, whatever the true derivative is. The null component's
4067            // shipped matrix (the rebuilt metric-consistent ridge) is EXACTLY
4068            // ℓ-invariant, so its analytic jets are exactly zero and its FD is
4069            // pure noise — measured at 3.5e-13 against a `1e-12` scale floor
4070            // that predates the exact answer. Grading that against a relative
4071            // tolerance alone asserts the ORACLE is exact, which it is not.
4072            // The factor 8 covers the handful of roundings between the two
4073            // rebuilds; it is not a fudge on the gradient, which is still
4074            // graded relatively wherever the FD resolves anything.
4075            let entry_scale = [&plus, &minus, &at]
4076                .iter()
4077                .flat_map(|built| built.active_penalties[candidate].matrix.iter())
4078                .fold(0.0_f64, |acc, value| acc.max(value.abs()));
4079            let first_floor = 8.0 * f64::EPSILON * entry_scale / h;
4080            let second_floor = 8.0 * f64::EPSILON * entry_scale / (h * h);
4081            let first_scale = fd_penalty_first
4082                .iter()
4083                .fold(1e-12_f64, |acc, value| acc.max(value.abs()));
4084            let second_scale = fd_penalty_second
4085                .iter()
4086                .fold(1e-10_f64, |acc, value| acc.max(value.abs()));
4087            for (analytic, finite_difference) in derivs.penalties_first[0][candidate]
4088                .iter()
4089                .zip(fd_penalty_first.iter())
4090            {
4091                assert!(
4092                    (analytic - finite_difference).abs() <= 1e-4 * first_scale + first_floor,
4093                    "candidate {candidate} ∂S~/∂lnℓ: analytic {analytic:.6e} vs FD \
4094                     {finite_difference:.6e} (rel budget {:.3e}, oracle floor {first_floor:.3e})",
4095                    1e-4 * first_scale
4096                );
4097            }
4098            for (analytic, finite_difference) in derivs.penalties_second_diag[0][candidate]
4099                .iter()
4100                .zip(fd_penalty_second.iter())
4101            {
4102                assert!(
4103                    (analytic - finite_difference).abs() <= 5e-3 * second_scale + second_floor,
4104                    "candidate {candidate} ∂²S~/∂lnℓ²: analytic {analytic:.6e} vs FD \
4105                     {finite_difference:.6e} (rel budget {:.3e}, oracle floor {second_floor:.3e})",
4106                    5e-3 * second_scale
4107                );
4108            }
4109        }
4110    }
4111
4112    /// Quadrature nodes must be the mass-weighted cell barycenters
4113    /// (first-moment-exact lumping), with empty cells keeping their seed
4114    /// coordinates at zero mass.
4115    #[test]
4116    pub(crate) fn quadrature_nodes_are_cell_barycenters() {
4117        // Two tight groups around (0,0) and (10,10); a third seed far away
4118        // captures nothing.
4119        let data = array![
4120            [0.0, 0.2],
4121            [0.4, -0.2],
4122            [0.2, 0.0],
4123            [9.8, 10.1],
4124            [10.2, 9.9],
4125        ];
4126        let seeds = array![[0.1, 0.1], [10.0, 10.0], [-50.0, -50.0]];
4127        let (nodes, masses) =
4128            measure_jet_quadrature_nodes(data.view(), seeds.view()).expect("quadrature nodes");
4129        assert!((masses.sum() - 1.0).abs() <= 1e-15, "masses must sum to 1");
4130        assert!((masses[0] - 0.6).abs() <= 1e-15);
4131        assert!((masses[1] - 0.4).abs() <= 1e-15);
4132        assert_eq!(masses[2], 0.0);
4133        // Cell 0 barycenter = (0.2, 0.0).
4134        assert_eq!(nodes[(0, 0)], 0.2);
4135        assert_eq!(nodes[(0, 1)], 0.0);
4136        // Cell 1 barycenter = (10.0, 10.0), which is not a sampled row.
4137        assert_eq!(nodes[(1, 0)], 10.0);
4138        assert_eq!(nodes[(1, 1)], 10.0);
4139        // Empty cell keeps its seed coordinates.
4140        assert_eq!(nodes[(2, 0)], -50.0);
4141        assert_eq!(nodes[(2, 1)], -50.0);
4142    }
4143
4144    /// Freeze→replay: rebuilding from the first build's frozen transform and
4145    /// frozen quadrature must reproduce design and penalty bit-for-bit (the
4146    /// predict-path contract).
4147    #[test]
4148    pub(crate) fn build_replay_roundtrip_reproduces_design_and_penalty() {
4149        // A bent filament with a side cluster; multiscale opt-in so this
4150        // exercises the per-scale (spectral) replay path (#1116).
4151        let n = 140usize;
4152        let data = Array2::<f64>::from_shape_fn((n, 2), |(i, k)| {
4153            let t = i as f64 / (n as f64 - 1.0);
4154            if k == 0 {
4155                t * 3.0
4156            } else {
4157                0.5 * (t * 3.0).cos() + if i % 9 == 0 { 0.8 } else { 0.0 }
4158            }
4159        });
4160        let spec = MeasureJetBasisSpec {
4161            center_strategy: CenterStrategy::FarthestPoint { num_centers: 70 },
4162            multiscale: true,
4163            ..MeasureJetBasisSpec::default()
4164        };
4165        let first = build_measure_jet_basis(data.view(), &spec).expect("first build");
4166        let BasisMetadata::MeasureJet {
4167            centers,
4168            length_scale,
4169            eps_band,
4170            order_s,
4171            alpha,
4172            tau0,
4173            masses,
4174            support_means,
4175            penalty_normalization_scales,
4176            raw_penalty_normalization_scales,
4177            fused_penalty_normalization_scale,
4178            constraint_transform,
4179            ..
4180        } = &first.metadata
4181        else {
4182            panic!("measure-jet build must return MeasureJet metadata");
4183        };
4184        let replay_spec = MeasureJetBasisSpec {
4185            center_strategy: CenterStrategy::UserProvided(centers.clone()),
4186            order_s: *order_s,
4187            alpha: *alpha,
4188            tau0: *tau0,
4189            num_scales: eps_band.len(),
4190            // MeasureJet freezes its range STANDARDIZED and replays it
4191            // verbatim; the tag is what records that it is the odd family out.
4192            length_scale: length_scale.standardized_value(),
4193            double_penalty: spec.double_penalty,
4194            learn_length_scale: spec.learn_length_scale,
4195            multiscale: spec.multiscale,
4196            identifiability: MeasureJetIdentifiability::FrozenTransform {
4197                transform: constraint_transform.clone().expect("fit-time z"),
4198            },
4199            frozen_quadrature: Some(MeasureJetFrozenQuadrature {
4200                masses: masses.clone(),
4201                eps_band: eps_band.clone(),
4202                support_means: support_means.clone(),
4203                penalty_normalization_scales: penalty_normalization_scales.clone(),
4204                raw_penalty_normalization_scales: raw_penalty_normalization_scales.clone(),
4205                fused_penalty_normalization_scale: *fused_penalty_normalization_scale,
4206                sigma_coord: None,
4207            }),
4208        };
4209        // Per-level mode: one candidate per band scale plus the function-space
4210        // null component, and the count must survive replay bit-for-bit.
4211        assert_eq!(
4212            first.active_penalties.len(),
4213            eps_band.len() + 1,
4214            "per-level mode must emit one candidate per scale + null component"
4215        );
4216        let second = build_measure_jet_basis(data.view(), &replay_spec).expect("replay build");
4217        let x1 = first.design.to_dense();
4218        let x2 = second.design.to_dense();
4219        assert_eq!(x1.shape(), x2.shape());
4220        for (a, b) in x1.iter().zip(x2.iter()) {
4221            assert!((a - b).abs() <= 1e-12, "design replay drift: {a} vs {b}");
4222        }
4223        assert_eq!(first.active_penalties.len(), second.active_penalties.len());
4224        for (p1, p2) in first
4225            .active_penalties
4226            .iter()
4227            .zip(second.active_penalties.iter())
4228        {
4229            for (a, b) in p1.matrix.iter().zip(p2.matrix.iter()) {
4230                assert!((a - b).abs() <= 1e-12, "penalty replay drift: {a} vs {b}");
4231            }
4232        }
4233    }
4234}