Skip to main content

pounce_sens_core/
activity.rs

1//! Post-solve activity classification (the covariance/information
2//! roadmap's item 0, gh #362).
3//!
4//! Classifies every bounded variable and every finite-bounded inequality
5//! row of a converged barrier solve into one of five statuses, keyed on
6//! the ratio of barrier curvature to the model's own curvature:
7//!
8//! ```text
9//! r = Σ / q,   Σ = z/s summed over the sides that exist,
10//!              q = |H_ii|                        (variable)
11//!                  |∇dⱼᵀ H ∇dⱼ| / ‖∇dⱼ‖⁴         (inequality row)
12//! ```
13//!
14//! The row denominator carries the fourth power so that `r` is
15//! invariant to rescaling the row: `d → c·d` sends `Σ → Σ/c²` while
16//! the curvature along the unit normal is unchanged, and `‖∇d‖⁴`
17//! restores the balance. Equivalently, the geometric barrier weight
18//! `Σ‖∇d‖²` (distance to the surface is `d/‖∇d‖`, its conjugate
19//! multiplier `v‖∇d‖`) is measured against the curvature along the
20//! unit normal. Variable bounds are invariant as written. This also
21//! absorbs the solver's own per-row `d_scale`.
22//!
23//! `H` is the exact Lagrangian Hessian, so constraint curvature
24//! contributes to `q` alongside the objective's. For variables, `q`
25//! reads the Hessian DIAGONAL only, so purely off-diagonal coupling is
26//! invisible to it: `f = x₁x₂` with bounds on both variables reports
27//! `unidentified` on every bound even though the bound directions have
28//! well-defined curvature. Items 1-4 of the covariance roadmap inherit
29//! these semantics where they consume the per-coordinate statuses;
30//! their reduced-block classification is where coupling becomes
31//! visible, folded into the reduced diagonal by elimination.
32//!
33//! # The diagonal is not the curvature that generates the multiplier
34//!
35//! The same DIAGONAL-only reading has a consequence sharper than a
36//! missed `q`, and it is the one to know before building on a status
37//! (gh#763). At a kink the multiplier is generated by the curvature
38//! **reduced** along the coordinate — what is left after the other
39//! free variables re-optimize — not by `H_ii`. Eliminating a free
40//! partner `y` from `[[h, c], [c, m]]` leaves `h − c²/m`, and
41//! `Σ = z/s` equals exactly that, so
42//!
43//! ```text
44//! r = reduced / diagonal
45//! ```
46//!
47//! which is `1` only where the coordinate is **decoupled**. Couple it
48//! and a genuine kink drops out of the `[1e-1, 1e1]` band and reads
49//! [`AMBIGUOUS`] — at any tolerance, because `r` there is
50//! `μ`-independent, so re-solving tighter reports the same thing. On a
51//! collocation model coupling between neighbouring coordinates is the
52//! normal case, not a corner.
53//!
54//! So **[`AMBIGUOUS`] is not "probably not a kink"**, and the class
55//! must not be used as a proxy for kink-ness. That inference is not
56//! hypothetical: gh#756 made it and shipped a first-order wrong
57//! derivative. [`reduced_activity`] is the accessor that answers the
58//! question the class does not — one back-solve per coordinate,
59//! normalizing by the reduced curvature — and on it the same kink
60//! reads [`WEAKLY_ACTIVE`] at every coupling.
61//!
62//! The default stays the diagonal because the reduced normalizer is
63//! the reciprocal diagonal of an *inverse*: there is no
64//! diagonal-of-the-inverse shortcut, so classifying every bounded
65//! variable that way is `n` back-solves, and at 62k variables that is
66//! not a post-solve diagnostic any more. The refinement is on demand,
67//! over the entries in question.
68//!
69//! The row path normalizes by a directional curvature rather than a
70//! diagonal, and `∇dᵀH∇d/‖∇d‖²` is a genuine curvature along the
71//! row's own gradient — strictly better than a bare `H_ii`, which is
72//! why it was not the one gh#763 fixed. But it is not *reduced*
73//! either: the other free coordinates still re-optimize. So a row's
74//! `r` is `reduced/directional` by the same algebra, `1` only where
75//! the row's direction is decoupled from the remaining free space,
76//! and a coupled row kink reads [`AMBIGUOUS`] at any tolerance for
77//! the same `μ`-independent reason (gh#804).
78//! [`reduced_row_activity`] is the row half of the answer, one
79//! back-solve per row: the row's own value IS a coordinate of the KKT
80//! system — the slack the barrier acts on, tied to the model by
81//! `dⱼ(x) = sⱼ` — so it is the same back-solve one block over.
82//!
83//! `r` is `O(μ)` when the bound is inactive, `O(1)` when weakly active
84//! (slack and multiplier vanish together), and `O(1/μ)` when strongly
85//! active, so one ratio separates the regimes at any `μ` where a fixed
86//! threshold on the slack or the multiplier alone cannot: both are
87//! `O(√μ)` at weak activity, so any constant tracks the solve rather
88//! than the geometry.
89//!
90//! Everything read here comes from the engine through
91//! [`BarrierIterate::activity_data`]: the bound multipliers on the
92//! iterate, the engine's own slacks, the barrier diagonal `Σ`, the
93//! barrier parameter, and the exact Lagrangian Hessian, so `H` is never
94//! recovered from the barrier-augmented factor. pounce's interior-point
95//! solver reports the declared-frame diagonal there when the held
96//! iterate came from crossover (gh#654). The reduced classifications
97//! add one back-solve per entry through [`crate::backsolver::SensBacksolver::solve_many`].
98//!
99//! The report is indexed in **user space**: `var_*` arrays have the
100//! user TNLP's full variable count and `row_*` arrays its full
101//! constraint count. A variable removed internally by
102//! `fixed_variable_treatment = make_parameter` (`lb == ub`, the
103//! default) reports [`FIXED`] at its own user index, and an equality
104//! constraint reports [`EQUALITY`], so user indices never shift.
105
106use std::rc::Rc;
107
108use pounce_common::types::Number;
109use pounce_linalg::Matrix;
110use pounce_linalg::dense_vector::{DenseVector, DenseVectorSpace};
111use pounce_linalg::triplet::{GenTMatrix, SymTMatrix};
112
113use crate::barrier::{ActivityData, BarrierIterate, EngineScaling, ModelMaps};
114
115// The status codes and the classification rule live in `activity_kernel`,
116// so the convex arm decides what a kink is with the same code rather than a
117// parallel one. Re-exported here so a caller of this module has them at hand.
118pub use crate::activity_kernel::{
119    AMBIGUOUS, EQUALITY, FIXED, INACTIVE, STRONGLY_ACTIVE, UNBOUNDED, UNIDENTIFIED, WEAKLY_ACTIVE,
120};
121use crate::activity_kernel::{
122    Entry, NOT_CLASSIFIED, classify_entry, off_path, sign_of, zero_gradient_row,
123};
124
125/// Per-variable and per-row classification of a converged solve.
126///
127/// All vectors are **user-space**: `var_*` have length `n_full_x` (the
128/// user TNLP's `n`) and `row_*` length `n_full_g` (the user's `m`).
129/// Entries with no finite bound hold [`UNBOUNDED`]; [`FIXED`]
130/// variables and [`EQUALITY`] rows are placeholders for entries the
131/// barrier never classified. All three carry `NaN` ratios.
132pub struct ActivityReport {
133    /// Barrier parameter of the converged iterate.
134    pub mu: Number,
135    /// Status per user variable (codes above).
136    ///
137    /// [`AMBIGUOUS`] here includes genuine kinks whose coordinate is
138    /// coupled to a neighbour: `q` is the Hessian diagonal, not the
139    /// reduced curvature that generates the multiplier, so the ratio
140    /// is `reduced/diagonal` (gh#763). Do not read the class as an
141    /// answer to "is this bound at a kink" — [`reduced_activity`]
142    /// answers that, one back-solve per coordinate.
143    pub var_status: Vec<i8>,
144    /// `Σ_i / q_i` per user variable; `NaN` where not classified.
145    /// For an [`UNIDENTIFIED`] entry the value is `Σ/floor`, a lower
146    /// bound on any honest ratio rather than the ratio itself, since
147    /// `q` is below the identification floor there.
148    ///
149    /// `q` is `|H_ii|`, so at a kink this ratio is
150    /// `reduced/diagonal` — `1` only where the coordinate is
151    /// decoupled, and `μ`-independent, so a tighter solve does not
152    /// move it. See [`Self::var_status`].
153    pub var_ratio: Vec<Number>,
154    /// Sign of the signed curvature `H_ii` (−1, 0, +1); the absolute
155    /// value goes into `q`, so an indefinite direction is reported
156    /// rather than hidden.
157    pub var_q_sign: Vec<i8>,
158    /// `s·z` differs from `μ` by more than a factor of ten on some
159    /// side: off the central path, or the bound was relaxed.
160    pub var_off_central_path: Vec<bool>,
161    /// Classified inactive yet `r` non-negligible: barrier curvature
162    /// where none should be.
163    pub var_contaminated: Vec<bool>,
164    /// The barrier diagonal `Σ_i = z/s` itself per user variable, both
165    /// sides summed; 0 where not classified. In **natural (unscaled)
166    /// units**, the repo's sensitivity-output contract: classification
167    /// runs on the solver's scaled quantities (the ratio is
168    /// scale-invariant), the report does not. The covariance roadmap's
169    /// item 1 subtracts exactly this from the factor's natural-units
170    /// reduced Hessian.
171    pub var_sigma: Vec<Number>,
172    /// Status per user constraint row.
173    ///
174    /// [`AMBIGUOUS`] here includes genuine kinks whose direction is
175    /// coupled to the remaining free space: `q` is the curvature
176    /// along the row's own gradient, not the reduced curvature that
177    /// generates the multiplier, so the ratio is
178    /// `reduced/directional` (gh#804). Do not read the class as an
179    /// answer to "is this row at a kink" — [`reduced_row_activity`]
180    /// answers that, one back-solve per row.
181    pub row_status: Vec<i8>,
182    /// `Σ_j / q_j` per user row; `NaN` where not classified.
183    /// [`UNIDENTIFIED`] entries hold `Σ/floor` as for variables.
184    ///
185    /// `q` is the directional curvature `|∇dᵀH∇d|/‖∇d‖²`, so at a
186    /// kink this ratio is `reduced/directional` — `1` only where the
187    /// row's direction is decoupled, and `μ`-independent, so a
188    /// tighter solve does not move it. See [`Self::row_status`].
189    pub row_ratio: Vec<Number>,
190    /// Sign of the signed row curvature `∇dⱼᵀ H ∇dⱼ`.
191    pub row_q_sign: Vec<i8>,
192    /// Central-path check per row, as for variables.
193    pub row_off_central_path: Vec<bool>,
194    /// Contamination check per row, as for variables.
195    pub row_contaminated: Vec<bool>,
196    /// The row barrier diagonal `Σ_j = v/s` per user row, both sides
197    /// summed; 0 where not classified. In **natural (unscaled) units**
198    /// like [`Self::var_sigma`], and RAW rather than the geometric
199    /// weight the classification uses: item 1 restricts the normal to
200    /// its own fitted block and applies its own `‖a‖²` factor there.
201    pub row_sigma: Vec<Number>,
202}
203
204/// Scatter a compressed (bounded-entries-only) vector to full length
205/// through the positions of its bounds. Entries without that bound stay 0.
206fn expand(compressed: &[Number], pos: &[usize], n: usize) -> Vec<Number> {
207    let mut full = vec![0.0; n];
208    for (k, &p) in pos.iter().enumerate() {
209        full[p] = compressed[k];
210    }
211    full
212}
213
214/// Presence mask for a bound side, from the same positions.
215fn present(pos: &[usize], n: usize) -> Vec<bool> {
216    let mut mask = vec![false; n];
217    for &p in pos {
218        mask[p] = true;
219    }
220    mask
221}
222
223/// What every classification reads from the engine: the converged
224/// quantities, the model maps and the scaling, each checked against the
225/// engine's block lengths before anything indexes with them.
226struct EngineRead {
227    data: ActivityData,
228    maps: ModelMaps,
229    scaling: EngineScaling,
230}
231
232fn read_engine<E: BarrierIterate + ?Sized>(bs: &E) -> Result<EngineRead, String> {
233    let data = bs.activity_data()?;
234    data.check()?;
235    let dims = bs.block_dims();
236    if data.sigma_x.len() != dims[0] || data.sigma_s.len() != dims[1] {
237        return Err(format!(
238            "activity data: {} variables and {} inequality rows, but the KKT blocks hold {} and {}",
239            data.sigma_x.len(),
240            data.sigma_s.len(),
241            dims[0],
242            dims[1]
243        ));
244    }
245    let maps = bs.model_maps()?;
246    maps.check(&dims)?;
247    let scaling = bs.scaling();
248    scaling.check(&dims)?;
249    Ok(EngineRead {
250        data,
251        maps,
252        scaling,
253    })
254}
255
256/// The exact Hessian diagonal: one pass over the triplet structure for
257/// the type `eval_h` builds today. The mat-vec fallback keeps any
258/// future non-triplet `SymMatrix` correct, at O(n·nnz) cost.
259fn hessian_diagonal(hess: &Rc<dyn pounce_linalg::SymMatrix>, n: usize) -> Vec<Number> {
260    let mut diag = vec![0.0; n];
261    if let Some(t) = hess.as_any().downcast_ref::<SymTMatrix>() {
262        // triplet indices are 1-based (the GenTMatrix convention);
263        // duplicates accumulate, matching mult_vector
264        for ((&i, &j), &v) in t.irows().iter().zip(t.jcols()).zip(t.values()) {
265            if i == j {
266                diag[(i - 1) as usize] += v;
267            }
268        }
269        return diag;
270    }
271    let space = DenseVectorSpace::new(n as i32);
272    let mut e = DenseVector::new(space.clone());
273    let mut he = DenseVector::new(space);
274    for (i, d) in diag.iter_mut().enumerate() {
275        e.values_mut().fill(0.0);
276        e.values_mut()[i] = 1.0;
277        he.values_mut().fill(0.0);
278        hess.mult_vector(1.0, &e, 0.0, &mut he);
279        // values_mut, not values: a zero product may have left the
280        // output homogeneous (empty backing slice); this materializes
281        *d = he.values_mut()[i];
282    }
283    diag
284}
285
286/// The per-variable pieces both classifiers measure against, with the
287/// `user-scaling` change of variables (gh#486 stage 3) divided out and
288/// the objective scale `df` still in: `Σ̃·d²` and `H̃_ii·d² = df·H_ii`.
289/// The ratio of the two is invariant to both, so classification runs
290/// here; the identification `floor` is not (it is one number shared
291/// across entries), which is why the change of variables comes out
292/// before it is formed.
293///
294/// Extracted so [`compute`] and [`reduced_activity`] cannot drift on
295/// the frame conversion — the dimension leg 1 of
296/// `sens_invariance_legs.rs` exists for.
297struct VarFrame {
298    /// Barrier diagonal `Σ_i` per var-x column.
299    sigma: Vec<Number>,
300    /// Exact-Lagrangian-Hessian diagonal per var-x column.
301    diag: Vec<Number>,
302    /// Identification floor shared by every entry, relative to the
303    /// largest curvature anywhere on the diagonal rather than just the
304    /// bounded entries, so a row-only model still measures `q` against
305    /// the model's own scale.
306    floor: Number,
307}
308
309fn var_frame(data: &ActivityData, d_var: Option<&[Number]>, n: usize) -> VarFrame {
310    let dv = |i: usize| -> Number { d_var.map_or(1.0, |d| d[i]) };
311    let hess = &data.hessian;
312    // `Σ̃_i = df·Σ_i/d_i²`: the `d_i²` comes out here, the `df` at the
313    // caller's export boundary (it cancels in every ratio, so
314    // classification never sees it).
315    let sigma = data
316        .sigma_x
317        .iter()
318        .enumerate()
319        .map(|(i, &s)| s * dv(i) * dv(i))
320        .collect();
321    let diag: Vec<Number> = hessian_diagonal(hess, n)
322        .iter()
323        .enumerate()
324        .map(|(i, &h)| h * dv(i) * dv(i))
325        .collect();
326    let max_abs_diag = diag.iter().fold(0.0, |a: Number, d| a.max(d.abs()));
327    let floor = Number::EPSILON.sqrt() * max_abs_diag.max(1.0);
328    VarFrame { sigma, diag, floor }
329}
330
331/// Classifies every bounded variable and every finite-bounded inequality row
332/// of the engine's converged point. See the module documentation for the
333/// rule.
334///
335/// The slacks have to be measured against the bounds the model declares,
336/// which means the engine must not have relaxed them (`bound_relax_factor =
337/// 0` in Ipopt and pounce, where the default is `1e-8`). A relaxed bound
338/// leaves an active slack near the relaxation rather than near `μ/z`, so the
339/// complementarity products no longer track `μ` and the ratios land in the
340/// wrong band. pounce's `Solver::classify_activity` refuses such a solve. The
341/// trait cannot report a relaxation, so an engine outside pounce checks it
342/// before calling this.
343pub fn compute<E: BarrierIterate + ?Sized>(bs: &E) -> Result<ActivityReport, String> {
344    let EngineRead {
345        data,
346        maps,
347        scaling,
348    } = read_engine(bs)?;
349    let mu = bs.barrier_mu();
350    let (n, m_d) = (data.sigma_x.len(), data.sigma_s.len());
351    let (obj_scale, d_scale) = (scaling.objective, scaling.inequality_rows.as_deref());
352
353    // Per-variable factors of a `user-scaling` change of variables
354    // (gh#486 stage 3), in var-x space; 1.0 everywhere when none ran.
355    // Every internal x-space quantity below is a `d`-transform of the
356    // model's own — writing `a_j` for the gradient of inequality row
357    // `j`, since `d` is spoken for here: `ã = a ⊘ d`,
358    // `H̃ = H ⊘ (d ⊗ d)`, `Σ̃ = Σ · df ⊘ (d ⊙ d)`. Undoing that here rather than only on
359    // the exported `Σ` is what keeps a status from depending on the
360    // conditioning the user asked for: the per-entry ratio `Σ/q` is
361    // invariant, but the identification `floor` is a single number
362    // shared across entries, so a non-uniform `d` moves entries across
363    // it. `1.0` multiplies are exact, so an unscaled solve is
364    // bit-identical to the pre-#486 path.
365    let d_var = scaling.variables.as_deref();
366    let dv = |i: usize| -> Number { d_var.map_or(1.0, |d| d[i]) };
367
368    // --- variables, in internal space ------------------------------------
369    let has_l = present(&data.x_l.pos, n);
370    let has_u = present(&data.x_u.pos, n);
371    let z_l = expand(&data.x_l.mult, &data.x_l.pos, n);
372    let z_u = expand(&data.x_u.mult, &data.x_u.pos, n);
373    // The solver's own slacks, deliberately, even when `Σ` below comes
374    // from the declared frame (gh#654): these feed `off_path` only, and
375    // "is `s·z` near `μ`" is a question about the central path, which is
376    // the barrier's geometry and therefore the barrier's slacks. A
377    // crossed-over point is off that path by construction and reads so
378    // under either frame.
379    let s_l = expand(&data.x_l.slack, &data.x_l.pos, n);
380    let s_u = expand(&data.x_u.slack, &data.x_u.pos, n);
381    // `Σ` and the Hessian diagonal, with the change of variables
382    // already divided out and the objective scale still in; the `df`
383    // comes off on export below (it cancels in every ratio, so
384    // classification never sees it).
385    let hess = &data.hessian;
386    let VarFrame {
387        sigma: sigma_x,
388        diag,
389        floor,
390    } = var_frame(&data, d_var, n);
391
392    let mut vars = vec![NOT_CLASSIFIED; n];
393    for i in 0..n {
394        if !(has_l[i] || has_u[i]) {
395            continue;
396        }
397        let mut e = classify_entry(sigma_x[i], diag[i], floor, mu);
398        e.off_path = (has_l[i] && off_path(s_l[i], z_l[i], mu))
399            || (has_u[i] && off_path(s_u[i], z_u[i], mu));
400        // the ratio is scale-invariant, so classification ran in the
401        // solver's own space up to the change of variables already
402        // divided out of `sigma_x` / `diag` above; the REPORTED sigma
403        // follows the repo's natural-units contract, and what is left
404        // to undo is the objective scale the internal z carries
405        e.sigma /= obj_scale;
406        vars[i] = e;
407    }
408
409    // --- inequality rows, in internal space -------------------------------
410    let rhas_l = present(&data.d_l.pos, m_d);
411    let rhas_u = present(&data.d_u.pos, m_d);
412    let v_l = expand(&data.d_l.mult, &data.d_l.pos, m_d);
413    let v_u = expand(&data.d_u.mult, &data.d_u.pos, m_d);
414    let rs_l = expand(&data.d_l.slack, &data.d_l.pos, m_d);
415    let rs_u = expand(&data.d_u.slack, &data.d_u.pos, m_d);
416    let sigma_s = &data.sigma_s;
417
418    let jac_d = &data.jac_d;
419    // One pass over the Jacobian triplets gathers every row's support
420    // and one pass over the Hessian triplets builds an adjacency view,
421    // so each row's curvature costs its own support times its
422    // neighbours instead of a full mat-vec pair per row (second
423    // review). The mat-vec loop below remains the fallback for any
424    // future non-triplet matrix types.
425    let mut rows = vec![NOT_CLASSIFIED; m_d];
426    let fast = match (
427        jac_d.as_any().downcast_ref::<GenTMatrix>(),
428        hess.as_any().downcast_ref::<SymTMatrix>(),
429    ) {
430        (Some(jt), Some(ht)) => {
431            // gather and merge each row's entries (triplet duplicates
432            // sum, matching mult_vector; indices are 1-based)
433            let mut support: Vec<Vec<(usize, Number)>> = vec![Vec::new(); m_d];
434            for ((&r, &c), &v) in jt.irows().iter().zip(jt.jcols()).zip(jt.values()) {
435                let col = (c - 1) as usize;
436                // `a = ã ⊙ d`: the row's own scale stays (the ratio
437                // divides it out), the change of variables does not.
438                support[(r - 1) as usize].push((col, v * dv(col)));
439            }
440            for sup in &mut support {
441                sup.sort_unstable_by_key(|&(c, _)| c);
442                sup.dedup_by(|a, b| {
443                    if a.0 == b.0 {
444                        b.1 += a.1;
445                        true
446                    } else {
447                        false
448                    }
449                });
450            }
451            let mut adj: Vec<Vec<(usize, Number)>> = vec![Vec::new(); n];
452            for ((&i, &l), &v) in ht.irows().iter().zip(ht.jcols()).zip(ht.values()) {
453                let (a, b) = ((i - 1) as usize, (l - 1) as usize);
454                // `H = H̃ ⊙ (d ⊗ d)`, matching the `d²` already taken
455                // out of `diag` (which sets the shared floor).
456                let v = v * dv(a) * dv(b);
457                adj[a].push((b, v));
458                if a != b {
459                    adj[b].push((a, v));
460                }
461            }
462            let mut scratch = vec![0.0; n];
463            for j in 0..m_d {
464                if !(rhas_l[j] || rhas_u[j]) {
465                    continue;
466                }
467                let sup = &support[j];
468                let norm2: Number = sup.iter().map(|&(_, g)| g * g).sum();
469                rows[j] = if norm2 <= 0.0 {
470                    zero_gradient_row(sigma_s[j], floor)
471                } else {
472                    for &(k, g) in sup {
473                        scratch[k] = g;
474                    }
475                    let mut ghg = 0.0;
476                    for &(k, gk) in sup {
477                        let mut acc = 0.0;
478                        for &(l, v) in &adj[k] {
479                            acc += v * scratch[l];
480                        }
481                        ghg += gk * acc;
482                    }
483                    for &(k, _) in sup {
484                        scratch[k] = 0.0;
485                    }
486                    // Σ·‖∇d‖² against curvature along the unit
487                    // normal: invariant to rescaling the row; the
488                    // report keeps the raw Σ
489                    let mut e = classify_entry(sigma_s[j] * norm2, ghg / norm2, floor, mu);
490                    e.sigma = sigma_s[j];
491                    e
492                };
493            }
494            true
495        }
496        _ => false,
497    };
498    if !fast {
499        let mspace = DenseVectorSpace::new(m_d as i32);
500        let mut e_row = DenseVector::new(mspace);
501        let nspace = DenseVectorSpace::new(n as i32);
502        let mut grad = DenseVector::new(nspace.clone());
503        let mut hgrad = DenseVector::new(nspace);
504        for j in 0..m_d {
505            if !(rhas_l[j] || rhas_u[j]) {
506                continue;
507            }
508            // ∇dⱼ = Jdᵀ eⱼ, then the curvature along the normal;
509            // values_mut throughout because a zero product may leave
510            // the output homogeneous (empty backing slice)
511            e_row.values_mut().fill(0.0);
512            e_row.values_mut()[j] = 1.0;
513            grad.values_mut().fill(0.0);
514            jac_d.trans_mult_vector(1.0, &e_row, 0.0, &mut grad);
515            // `a = ã ⊙ d`, then `aᵀHa = uᵀH̃u` with `u = d ⊙ a`
516            // (since `H = H̃ ⊙ (d ⊗ d)`) — so the vector handed to the
517            // internal Hessian carries `d²`, and `norm2` carries `d`.
518            let norm2: Number = grad
519                .values_mut()
520                .iter()
521                .enumerate()
522                .map(|(i, g)| (*g * dv(i)) * (*g * dv(i)))
523                .sum();
524            rows[j] = if norm2 <= 0.0 {
525                zero_gradient_row(sigma_s[j], floor)
526            } else {
527                for (i, g) in grad.values_mut().iter_mut().enumerate() {
528                    *g *= dv(i) * dv(i);
529                }
530                hgrad.values_mut().fill(0.0);
531                hess.mult_vector(1.0, &grad, 0.0, &mut hgrad);
532                let ghg: Number = {
533                    let h = hgrad.values_mut();
534                    grad.values_mut()
535                        .iter()
536                        .zip(h.iter())
537                        .map(|(g, h)| g * h)
538                        .sum()
539                };
540                let mut e = classify_entry(sigma_s[j] * norm2, ghg / norm2, floor, mu);
541                e.sigma = sigma_s[j];
542                e
543            };
544        }
545    }
546    for j in 0..m_d {
547        if !(rhas_l[j] || rhas_u[j]) {
548            continue;
549        }
550        rows[j].off_path = (rhas_l[j] && off_path(rs_l[j], v_l[j], mu))
551            || (rhas_u[j] && off_path(rs_u[j], v_u[j], mu));
552        // natural-units report, as for variables: the scaled row
553        // multiplier carries df/dg and the scaled slack dg, so
554        // Sigma_nat = Sigma * dg^2 / df
555        let dg = d_scale.map_or(1.0, |v| v[j]);
556        rows[j].sigma *= dg * dg / obj_scale;
557    }
558
559    // --- scatter to user space --------------------------------------------
560    let n_full_x = maps.n_full_x;
561    let n_full_g = maps.n_full_g();
562
563    let fixed_entry = Entry {
564        status: FIXED,
565        ..NOT_CLASSIFIED
566    };
567    let mut var_full = vec![fixed_entry; n_full_x];
568    for (i, e) in vars.iter().enumerate() {
569        var_full[maps.var_x_to_full_x[i]] = *e;
570    }
571
572    let equality_entry = Entry {
573        status: EQUALITY,
574        ..NOT_CLASSIFIED
575    };
576    let mut row_full = vec![equality_entry; n_full_g];
577    for (slot, d_pos) in row_full.iter_mut().zip(&maps.full_g_to_d) {
578        if let Some(d) = d_pos {
579            *slot = rows[*d];
580        }
581    }
582
583    Ok(ActivityReport {
584        mu,
585        var_status: var_full.iter().map(|e| e.status).collect(),
586        var_ratio: var_full.iter().map(|e| e.ratio).collect(),
587        var_q_sign: var_full.iter().map(|e| e.q_sign).collect(),
588        var_off_central_path: var_full.iter().map(|e| e.off_path).collect(),
589        var_contaminated: var_full.iter().map(|e| e.contaminated).collect(),
590        var_sigma: var_full.iter().map(|e| e.sigma).collect(),
591        row_status: row_full.iter().map(|e| e.status).collect(),
592        row_ratio: row_full.iter().map(|e| e.ratio).collect(),
593        row_q_sign: row_full.iter().map(|e| e.q_sign).collect(),
594        row_off_central_path: row_full.iter().map(|e| e.off_path).collect(),
595        row_contaminated: row_full.iter().map(|e| e.contaminated).collect(),
596        row_sigma: row_full.iter().map(|e| e.sigma).collect(),
597    })
598}
599
600/// One coordinate's activity re-measured against the curvature that
601/// actually generates its multiplier, rather than against the Hessian
602/// diagonal (gh#763).
603///
604/// Parallel arrays, one entry per requested variable, in the order
605/// they were requested. See [`reduced_activity`] for what the
606/// quantities mean.
607#[derive(Debug, Clone)]
608pub struct ReducedActivityReport {
609    /// Barrier parameter of the converged iterate, as
610    /// [`ActivityReport::mu`].
611    pub mu: Number,
612    /// The user-space variable index each entry answers about.
613    pub var: Vec<usize>,
614    /// Status from the same rule [`ActivityReport::var_status`] uses,
615    /// applied to [`Self::ratio`].
616    pub status: Vec<i8>,
617    /// `Σ_i / |q_i^red|`. `NaN` where nothing was classified.
618    pub ratio: Vec<Number>,
619    /// The reduced curvature `q_i^red` itself, signed, in **natural
620    /// (unscaled) units** like [`ActivityReport::var_sigma`]. `NaN`
621    /// for a [`FIXED`] variable and where the factor offers no
622    /// reduced curvature (see [`reduced_activity`]).
623    pub q_reduced: Vec<Number>,
624    /// Sign of [`Self::q_reduced`] (−1, 0, +1); the absolute value is
625    /// what the ratio divides by.
626    pub q_sign: Vec<i8>,
627    /// `Σ_i`, natural units, identical to the same variable's entry in
628    /// [`ActivityReport::var_sigma`].
629    pub sigma: Vec<Number>,
630}
631
632/// Why a [`reduced_activity`] call could not be answered.
633#[derive(Debug, Clone, PartialEq)]
634pub enum ReducedActivityError {
635    /// A requested index is not a user variable. Carries the offending
636    /// index and the user TNLP's `n`.
637    OutOfRange { got: usize, n_full_x: usize },
638    /// The back-solve against the held factor failed.
639    Backsolve,
640    /// The engine could not supply the converged quantities, or supplied
641    /// inconsistent ones. The message says which.
642    Data(String),
643}
644
645/// [`compute`]'s per-variable classification, re-normalized by the
646/// **reduced** curvature along each requested coordinate instead of
647/// the Hessian diagonal — one back-solve against the held factor per
648/// variable asked about (gh#763).
649///
650/// # What is different
651///
652/// [`compute`] forms a variable's ratio as `Σ_i / |H_ii|`. At a kink
653/// the multiplier is not generated by the diagonal: it is generated by
654/// the curvature **reduced** along that coordinate, i.e. what is left
655/// after the other free variables re-optimize. Eliminating a free
656/// partner `y` from `[[h, c], [c, m]]` leaves `h − c²/m`, and
657/// `Σ = z/s` equals that, so the diagonal ratio is `1` only when the
658/// coordinate is **decoupled** (`c = 0`). Couple it and a genuine kink
659/// reads [`AMBIGUOUS`] at any tolerance, because the ratio is
660/// `μ`-independent — routine on a collocation model, where coupling
661/// between neighbouring coordinates is the normal case. This function
662/// is the answer to "is that AMBIGUOUS entry a kink?": on the reduced
663/// normalizer the same kink reads [`WEAKLY_ACTIVE`] whatever it is
664/// coupled to.
665///
666/// # How it is computed
667///
668/// The reduced curvature the barrier subproblem sees along coordinate
669/// `i` is the reciprocal of the `i`-th diagonal entry of the inverse
670/// of the barrier-augmented KKT matrix — one column of `K⁻¹`, so one
671/// back-solve — with the barrier's own contribution at `i` taken back
672/// off:
673///
674/// ```text
675/// q_i^red = 1 / (K⁻¹)_ii − Σ_i
676/// ```
677///
678/// Adding `diag(Σ)` to the free block shifts entry `i` by exactly
679/// `Σ_i`, which the subtraction removes; it also shifts every OTHER
680/// free coordinate, which is deliberately kept. That is what makes the
681/// answer the curvature the multiplier is generated against: a
682/// strongly active neighbour carries `Σ = O(1/μ)` and so does not
683/// re-optimize, an inactive one carries `O(μ)` and re-optimizes
684/// freely, and the elimination weights each accordingly. Eliminating
685/// against the bare Hessian would let a pinned neighbour move.
686///
687/// `K` here is the natural-units KKT matrix, so `q_i^red` and `Σ_i`
688/// are both in the model's own units and the ratio is invariant to
689/// `nlp_scaling_method` and to a `user-scaling` change of variables,
690/// exactly as [`compute`]'s is.
691///
692/// # Cost, and why this is not the default
693///
694/// The correct normalizer is the reciprocal diagonal of an *inverse*,
695/// and there is no diagonal-of-the-inverse shortcut: it is one
696/// back-solve per coordinate. Classifying every bounded variable this
697/// way is `n` back-solves, which on a 62k-variable model is not a
698/// post-solve diagnostic any more. So [`compute`] keeps the `O(nnz)`
699/// diagonal and this is the on-demand refinement — the natural call is
700/// over the [`AMBIGUOUS`] entries of a report, which is a handful on
701/// the models where the question arises.
702///
703/// Pass those indices in ONE call: the back-solves batch, and the
704/// per-call fixed cost — one pass over the Hessian for the shared
705/// identification floor — is paid once rather than per index.
706///
707/// # Edge cases
708///
709/// * A [`FIXED`] variable has no column in the factor: status
710///   [`FIXED`], `NaN` curvature and ratio, as in the report.
711/// * A variable with no finite bound gets its `q_i^red` (it is
712///   well-defined and already paid for) but status [`UNBOUNDED`] and a
713///   `NaN` ratio: there is no bound question to answer.
714/// * A coordinate the constraints determine outright has
715///   `(K⁻¹)_ii = 0`: no direction left to reduce along, so `q_i^red`
716///   is infinite and the ratio is `0`, i.e. [`INACTIVE`] — whatever
717///   holds that coordinate, it is not its bound. A `(K⁻¹)_ii` at
718///   roundoff level reads the same from either side of zero.
719/// * A `NaN` from the back-solve reports [`UNIDENTIFIED`] with a `NaN`
720///   curvature and ratio.
721/// * `|q_i^red|` below the same identification floor [`compute`] uses
722///   reports [`UNIDENTIFIED`], as there.
723/// * Under `obj_scaling_factor < 0` — the documented way to maximize —
724///   `Sigma` and `q_i^red` are both reported with the sign the
725///   natural-units contract gives them (negative, as
726///   [`ActivityReport::var_sigma`] is), but the CLASSIFICATION runs on
727///   the objective-scale-positive orientation [`compute`] classifies
728///   in, so a status here means the same thing at either sign of `df`.
729///
730/// Variable bounds only. The row path carries the same un-reduced
731/// distinction — see [`reduced_row_activity`], which is this function
732/// one KKT block over (gh#804).
733pub fn reduced_activity<E: BarrierIterate + ?Sized>(
734    bs: &E,
735    user_vars: &[usize],
736) -> Result<ReducedActivityReport, ReducedActivityError> {
737    let EngineRead {
738        data,
739        maps,
740        scaling,
741    } = read_engine(bs).map_err(ReducedActivityError::Data)?;
742    let mu = bs.barrier_mu();
743    let n = data.sigma_x.len();
744    let full_x_to_var_x = maps.full_x_to_var_x();
745
746    // full-x in, var-x rows out, through the same map the report
747    // scatters through: reading the user index as a factor row returns
748    // a NEIGHBORING variable's answer wherever a fixed variable
749    // precedes it (the gh#450 hazard).
750    let n_full_x = maps.n_full_x;
751    let mut rows: Vec<Option<usize>> = Vec::with_capacity(user_vars.len());
752    for &i in user_vars {
753        if i >= n_full_x {
754            return Err(ReducedActivityError::OutOfRange { got: i, n_full_x });
755        }
756        rows.push(full_x_to_var_x[i]);
757    }
758
759    let obj_scale = scaling.objective;
760    let has_l = present(&data.x_l.pos, n);
761    let has_u = present(&data.x_u.pos, n);
762    let frame = var_frame(&data, scaling.variables.as_deref(), n);
763    // `Σ` and the floor come out of `var_frame` with the change of
764    // variables divided out but the objective scale still in; the
765    // reciprocal `(K⁻¹)_ii` below is natural units already, so both
766    // sides of the subtraction meet there.
767    let floor = frame.floor / obj_scale.abs();
768
769    // One unit RHS per requested variable, batched against the held
770    // factor. Chunked so the buffers stay bounded when a caller hands
771    // over a long list rather than the report's ambiguous entries.
772    const CHUNK: usize = 64;
773    let dim = bs.dim();
774    let solve_rows: Vec<usize> = rows.iter().flatten().copied().collect();
775    let mut kinv: Vec<Number> = Vec::with_capacity(solve_rows.len());
776    for chunk in solve_rows.chunks(CHUNK) {
777        let k = chunk.len();
778        let mut rhs = vec![0.0; k * dim];
779        let mut lhs = vec![0.0; k * dim];
780        for (c, &r) in chunk.iter().enumerate() {
781            rhs[c * dim + r] = 1.0;
782        }
783        if !bs.solve_many(&rhs, &mut lhs, k) {
784            return Err(ReducedActivityError::Backsolve);
785        }
786        for (c, &r) in chunk.iter().enumerate() {
787            kinv.push(lhs[c * dim + r]);
788        }
789    }
790
791    let mut out = ReducedActivityReport {
792        mu,
793        var: user_vars.to_vec(),
794        status: Vec::with_capacity(user_vars.len()),
795        ratio: Vec::with_capacity(user_vars.len()),
796        q_reduced: Vec::with_capacity(user_vars.len()),
797        q_sign: Vec::with_capacity(user_vars.len()),
798        sigma: Vec::with_capacity(user_vars.len()),
799    };
800    let mut next = 0usize;
801    for &row in &rows {
802        let Some(row) = row else {
803            out.status.push(FIXED);
804            out.ratio.push(Number::NAN);
805            out.q_reduced.push(Number::NAN);
806            out.q_sign.push(0);
807            out.sigma.push(0.0);
808            continue;
809        };
810        let d = kinv[next];
811        next += 1;
812        let sigma = frame.sigma[row] / obj_scale;
813        if !d.is_finite() {
814            out.status.push(UNIDENTIFIED);
815            out.ratio.push(Number::NAN);
816            out.q_reduced.push(Number::NAN);
817            out.q_sign.push(0);
818            out.sigma.push(sigma);
819            continue;
820        }
821        // `(K⁻¹)_ii = 0` — the constraints determine the coordinate
822        // outright, so there is no direction left to reduce along —
823        // sends `q` to an infinity the ratio divides to zero, i.e.
824        // INACTIVE: whatever holds the coordinate there, it is not the
825        // bound. A `(K⁻¹)_ii` at roundoff level lands in the same
826        // class from either side of zero, which is why no guard
827        // branches on its sign; a genuinely negative reduced curvature
828        // is modest in magnitude and reports through `q_sign`, exactly
829        // as an indefinite `H_ii` does in the report.
830        let q = 1.0 / d - sigma;
831        // classify in the same orientation `compute` does: it runs the
832        // rule on the df-in `Sigma` (internal `z/s`, non-negative),
833        // dividing the objective scale out only on export. Here both
834        // sides are already natural, so a NEGATIVE df -- the
835        // documented way to maximize -- would otherwise hand the rule
836        // a negative ratio and read a pinned bound as INACTIVE.
837        let sgn = if obj_scale < 0.0 { -1.0 } else { 1.0 };
838        let e = classify_entry(sigma * sgn, q * sgn, floor, mu);
839        // no finite bound: the curvature is still an answer, the
840        // activity question is not
841        let bounded = has_l[row] || has_u[row];
842        out.status.push(if bounded { e.status } else { UNBOUNDED });
843        out.ratio.push(if bounded { e.ratio } else { Number::NAN });
844        out.q_reduced.push(q);
845        out.q_sign.push(sign_of(q));
846        out.sigma.push(sigma);
847    }
848    debug_assert_eq!(next, kinv.len(), "every free row consumed its solve");
849    Ok(out)
850}
851
852/// One constraint row's activity re-measured against the curvature
853/// that actually generates its multiplier, rather than against the
854/// directional curvature along its own gradient (gh#804).
855///
856/// Parallel arrays, one entry per requested row, in the order they
857/// were requested. See [`reduced_row_activity`] for what the
858/// quantities mean.
859#[derive(Debug, Clone)]
860pub struct ReducedRowActivityReport {
861    /// Barrier parameter of the converged iterate, as
862    /// [`ActivityReport::mu`].
863    pub mu: Number,
864    /// The user-space constraint index each entry answers about.
865    pub row: Vec<usize>,
866    /// Status from the same rule [`ActivityReport::row_status`] uses,
867    /// applied to [`Self::ratio`].
868    pub status: Vec<i8>,
869    /// `Σ_j‖∇dⱼ‖² / |q_j^red|`. `NaN` where nothing was classified.
870    ///
871    /// The numerator is the geometric barrier weight, not the raw
872    /// `Σ_j` [`Self::sigma`] reports — the same pairing
873    /// [`ActivityReport::row_ratio`] uses, so the two ratios are
874    /// directly comparable and agree on a decoupled row.
875    pub ratio: Vec<Number>,
876    /// The reduced curvature `q_j^red` itself, signed, along the
877    /// **unit** normal `∇dⱼ/‖∇dⱼ‖` and in **natural (unscaled)
878    /// units** — the same quantity and units
879    /// [`ActivityReport::row_q_sign`] takes the sign of, so it is what
880    /// `|∇dⱼᵀH∇dⱼ|/‖∇dⱼ‖²` would have been had the curvature been
881    /// reduced. `NaN` for an [`EQUALITY`] row and where the back-solve
882    /// offers no reduced curvature (see [`reduced_row_activity`]).
883    pub q_reduced: Vec<Number>,
884    /// Sign of [`Self::q_reduced`] (−1, 0, +1); the absolute value is
885    /// what the ratio divides by.
886    pub q_sign: Vec<i8>,
887    /// `Σ_j`, RAW (not the geometric weight the ratio uses) and in
888    /// natural units, identical to the same row's entry in
889    /// [`ActivityReport::row_sigma`].
890    pub sigma: Vec<Number>,
891}
892
893/// Why a [`reduced_row_activity`] call could not be answered.
894#[derive(Debug, Clone, PartialEq)]
895pub enum ReducedRowActivityError {
896    /// A requested index is not a user constraint. Carries the
897    /// offending index and the user TNLP's `m`.
898    OutOfRange { got: usize, n_full_g: usize },
899    /// The back-solve against the held factor failed.
900    Backsolve,
901    /// The engine could not supply the converged quantities, or supplied
902    /// inconsistent ones. The message says which.
903    Data(String),
904}
905
906/// `‖∇dⱼ‖²` for the requested internal inequality rows, in the frame
907/// [`compute`] classifies in: `a = ã ⊙ d`, so the change of variables
908/// is divided out and the row's own `d_scale` is still in. The caller
909/// divides that `dg²` out to reach natural units.
910///
911/// One pass over the Jacobian triplets for the whole batch, with the
912/// mat-vec loop kept as the fallback for any future non-triplet
913/// matrix type — the same two paths, and the same duplicate-summing
914/// convention, [`compute`] uses.
915fn row_norm2(
916    jac_d: &Rc<dyn Matrix>,
917    d_var: Option<&[Number]>,
918    wanted: &[Option<usize>],
919    n: usize,
920    m_d: usize,
921    out_len: usize,
922) -> Vec<Number> {
923    let dv = |i: usize| -> Number { d_var.map_or(1.0, |d| d[i]) };
924    let mut norm2 = vec![0.0; out_len];
925    if let Some(jt) = jac_d.as_any().downcast_ref::<GenTMatrix>() {
926        let mut support: Vec<Vec<(usize, Number)>> = vec![Vec::new(); out_len];
927        for ((&r, &c), &v) in jt.irows().iter().zip(jt.jcols()).zip(jt.values()) {
928            let Some(slot) = wanted[(r - 1) as usize] else {
929                continue;
930            };
931            let col = (c - 1) as usize;
932            support[slot].push((col, v * dv(col)));
933        }
934        for (slot, sup) in support.iter_mut().enumerate() {
935            // triplet duplicates sum before the square, matching
936            // `mult_vector` and `compute`'s own gather
937            sup.sort_unstable_by_key(|&(c, _)| c);
938            sup.dedup_by(|a, b| {
939                if a.0 == b.0 {
940                    b.1 += a.1;
941                    true
942                } else {
943                    false
944                }
945            });
946            norm2[slot] = sup.iter().map(|&(_, g)| g * g).sum();
947        }
948        return norm2;
949    }
950    let mspace = DenseVectorSpace::new(m_d as i32);
951    let mut e_row = DenseVector::new(mspace);
952    let nspace = DenseVectorSpace::new(n as i32);
953    let mut grad = DenseVector::new(nspace);
954    for (j, slot) in wanted.iter().enumerate() {
955        let Some(slot) = *slot else { continue };
956        // values_mut throughout: a zero product may leave the output
957        // homogeneous (empty backing slice)
958        e_row.values_mut().fill(0.0);
959        e_row.values_mut()[j] = 1.0;
960        grad.values_mut().fill(0.0);
961        jac_d.trans_mult_vector(1.0, &e_row, 0.0, &mut grad);
962        norm2[slot] = grad
963            .values_mut()
964            .iter()
965            .enumerate()
966            .map(|(i, g)| (*g * dv(i)) * (*g * dv(i)))
967            .sum();
968    }
969    norm2
970}
971
972/// [`compute`]'s per-row classification, re-normalized by the
973/// **reduced** curvature along each requested row's gradient instead
974/// of the directional curvature `∇dᵀH∇d/‖∇d‖²` — one back-solve
975/// against the held factor per row asked about (gh#804).
976///
977/// The row counterpart of [`reduced_activity`], and the same defect:
978/// gh#763 for rows.
979///
980/// # What is different
981///
982/// [`compute`] forms a row's ratio as `Σ_j‖∇d‖⁴/|∇dᵀH∇d|`. That
983/// denominator is a genuine curvature along the row's own gradient
984/// direction — strictly better than the variable path's bare `H_ii`,
985/// which is why gh#763 fixed the variables first — but it is not a
986/// *reduced* curvature: it does not account for the other free
987/// coordinates re-optimizing. The quantity that generates a row's
988/// multiplier is what is left after that elimination, exactly as for
989/// a variable, so a row's `r` is `reduced/directional` and equals `1`
990/// only where the row's direction is decoupled from the remaining
991/// free space. Couple it and a genuine row kink falls out of the
992/// `[1e-1, 1e1]` band and reads [`AMBIGUOUS`] — at any tolerance,
993/// because that ratio is `μ`-independent, so re-solving tighter
994/// reports the same thing.
995///
996/// # How it is computed
997///
998/// The row's own value IS a coordinate of the KKT system: the slack
999/// `s_j` the barrier acts on, tied to the model by `dⱼ(x) = s_j`. So
1000/// the reduced curvature the barrier subproblem sees along the row is
1001/// the reciprocal of that coordinate's diagonal entry of the inverse
1002/// — one back-solve against a unit right-hand side in the `s` block —
1003/// with the row's own barrier contribution taken back off:
1004///
1005/// ```text
1006/// q_j^raw = 1 / (K⁻¹)_{sⱼsⱼ} − Σ_j        (per unit of dⱼ)
1007/// q_j^red = q_j^raw · ‖∇dⱼ‖²              (per unit of x, reported)
1008/// ```
1009///
1010/// Driving the `x` block with the row's gradient `∇dⱼ` instead gives
1011/// the identical number — `∇dⱼᵀK⁻¹∇dⱼ = (K⁻¹)_{sⱼsⱼ}` for every `j`,
1012/// since `∇dⱼ` reaches the system only through the row it defines —
1013/// so the slack unit vector is used: it needs no gradient assembled
1014/// into the right-hand side, and it is the same call
1015/// [`reduced_activity`] makes one block over.
1016///
1017/// Everything the elimination weights is what makes this the
1018/// curvature the multiplier is generated against, exactly as in
1019/// [`reduced_activity`]: a strongly active neighbour carries
1020/// `Σ = O(1/μ)` and does not re-optimize, an inactive one carries
1021/// `O(μ)` and re-optimizes freely.
1022///
1023/// The `‖∇dⱼ‖²` puts the answer along the **unit** normal, which is
1024/// where [`compute`]'s `q` lives and what the shared identification
1025/// floor is scaled for. Paired with the geometric weight
1026/// `Σ_j‖∇dⱼ‖²` in the numerator — [`compute`]'s pairing — the ratio
1027/// is invariant to rescaling the row, so `d → c·d` does not change a
1028/// status here any more than it does there.
1029///
1030/// `K` is the natural-units KKT matrix, so the ratio is also
1031/// invariant to `nlp_scaling_method` and to a `user-scaling` change
1032/// of variables.
1033///
1034/// # Cost, and why this is not the default
1035///
1036/// One back-solve per row, for the same reason [`reduced_activity`]
1037/// costs one per variable: the correct normalizer is a diagonal entry
1038/// of an *inverse*, and there is no shortcut to it. So [`compute`]
1039/// keeps the `O(nnz)` directional curvature and this is the on-demand
1040/// refinement — the natural call is over the [`AMBIGUOUS`] rows of a
1041/// report. Pass them in ONE call: the back-solves batch, and the
1042/// per-call fixed cost (one pass over the Hessian for the shared
1043/// identification floor, one over the Jacobian for the norms) is paid
1044/// once rather than per row.
1045///
1046/// # Edge cases
1047///
1048/// * An equality row has no slack and no barrier multiplier pair:
1049///   status [`EQUALITY`], `NaN` curvature and ratio, as in the report.
1050/// * An inequality row with no finite bound gets its `q_j^red` but
1051///   status [`UNBOUNDED`] and a `NaN` ratio: no bound question to
1052///   answer.
1053/// * A row whose gradient vanishes at the iterate has no direction to
1054///   measure curvature along: [`UNIDENTIFIED`] with the raw
1055///   `Σ/floor` lower bound, exactly as [`compute`] reports it.
1056/// * A row the constraints determine outright has `(K⁻¹)_{sⱼsⱼ} = 0`:
1057///   no direction left to reduce along, so `q_j^red` is infinite and
1058///   the ratio is `0`, i.e. [`INACTIVE`] — whatever holds that row, it
1059///   is not its own bound.
1060/// * A `NaN` from the back-solve reports [`UNIDENTIFIED`] with a `NaN`
1061///   curvature and ratio.
1062/// * `|q_j^red|` below the same identification floor [`compute`] uses
1063///   reports [`UNIDENTIFIED`], as there.
1064/// * Under `obj_scaling_factor < 0` — the documented way to maximize —
1065///   `Σ` and `q_j^red` are both reported with the sign the
1066///   natural-units contract gives them, but the CLASSIFICATION runs on
1067///   the objective-scale-positive orientation [`compute`] classifies
1068///   in, so a status here means the same thing at either sign of `df`.
1069pub fn reduced_row_activity<E: BarrierIterate + ?Sized>(
1070    bs: &E,
1071    user_rows: &[usize],
1072) -> Result<ReducedRowActivityReport, ReducedRowActivityError> {
1073    let EngineRead {
1074        data,
1075        maps,
1076        scaling,
1077    } = read_engine(bs).map_err(ReducedRowActivityError::Data)?;
1078    let mu = bs.barrier_mu();
1079    let (n, m_d) = (data.sigma_x.len(), data.sigma_s.len());
1080
1081    // full-g in, d-block rows out, through the NLP's own c/d map:
1082    // reading the user index as an inequality position returns a
1083    // NEIGHBORING row's answer wherever an equality precedes it (the
1084    // gh#450 hazard, one block over). It is the same map
1085    // `Solver::d_multiplier_rows` addresses the `y_d` block with
1086    // (gh#910), deliberately rather than a second ascending scan that
1087    // agrees with it today: the gate that accepts a strictly active
1088    // inequality's `dλ/dp` classifies with THIS function and then
1089    // reads THAT row, so a disagreement between the two would classify
1090    // one row and answer about another.
1091    let n_full_g = maps.n_full_g();
1092    let d_index = &maps.full_g_to_d;
1093    let mut rows: Vec<Option<usize>> = Vec::with_capacity(user_rows.len());
1094    for &j in user_rows {
1095        if j >= n_full_g {
1096            return Err(ReducedRowActivityError::OutOfRange { got: j, n_full_g });
1097        }
1098        rows.push(d_index[j]);
1099    }
1100
1101    let (obj_scale, d_scale) = (scaling.objective, scaling.inequality_rows.as_deref());
1102    let rhas_l = present(&data.d_l.pos, m_d);
1103    let rhas_u = present(&data.d_u.pos, m_d);
1104    let sigma_s = &data.sigma_s;
1105
1106    // Which internal rows the batch asks about, and where each one's
1107    // norm lands. A row requested twice shares one slot and one gather.
1108    let mut slot_of: Vec<Option<usize>> = vec![None; m_d];
1109    let mut n_slots = 0usize;
1110    for row in rows.iter().flatten() {
1111        if slot_of[*row].is_none() {
1112            slot_of[*row] = Some(n_slots);
1113            n_slots += 1;
1114        }
1115    }
1116
1117    // the NLP borrow above is dropped: the Cq getters re-borrow it
1118    // mutably for lazy evaluation
1119    let floor = var_frame(&data, scaling.variables.as_deref(), n).floor;
1120    let norm2 = row_norm2(
1121        &data.jac_d,
1122        scaling.variables.as_deref(),
1123        &slot_of,
1124        n,
1125        m_d,
1126        n_slots,
1127    );
1128    let floor = floor / obj_scale.abs();
1129
1130    // One unit RHS per requested row, in the `s` block, batched
1131    // against the held factor. Chunked so the buffers stay bounded
1132    // when a caller hands over a long list rather than the report's
1133    // ambiguous rows.
1134    const CHUNK: usize = 64;
1135    let dim = bs.dim();
1136    let s_offset = bs.block_dims()[0];
1137    let solve_rows: Vec<usize> = rows.iter().flatten().map(|&r| s_offset + r).collect();
1138    let mut kinv: Vec<Number> = Vec::with_capacity(solve_rows.len());
1139    for chunk in solve_rows.chunks(CHUNK) {
1140        let k = chunk.len();
1141        let mut rhs = vec![0.0; k * dim];
1142        let mut lhs = vec![0.0; k * dim];
1143        for (c, &r) in chunk.iter().enumerate() {
1144            rhs[c * dim + r] = 1.0;
1145        }
1146        if !bs.solve_many(&rhs, &mut lhs, k) {
1147            return Err(ReducedRowActivityError::Backsolve);
1148        }
1149        for (c, &r) in chunk.iter().enumerate() {
1150            kinv.push(lhs[c * dim + r]);
1151        }
1152    }
1153
1154    let mut out = ReducedRowActivityReport {
1155        mu,
1156        row: user_rows.to_vec(),
1157        status: Vec::with_capacity(user_rows.len()),
1158        ratio: Vec::with_capacity(user_rows.len()),
1159        q_reduced: Vec::with_capacity(user_rows.len()),
1160        q_sign: Vec::with_capacity(user_rows.len()),
1161        sigma: Vec::with_capacity(user_rows.len()),
1162    };
1163    // `compute` runs the rule on the df-in `Sigma` (internal `v/s`,
1164    // non-negative), dividing the objective scale out only on export.
1165    // Here every quantity is already natural, so a NEGATIVE df -- the
1166    // documented way to maximize -- would otherwise hand the rule a
1167    // negative ratio and read a pinned row as INACTIVE.
1168    let sgn = if obj_scale < 0.0 { -1.0 } else { 1.0 };
1169    let mut next = 0usize;
1170    for &row in &rows {
1171        let Some(row) = row else {
1172            out.status.push(EQUALITY);
1173            out.ratio.push(Number::NAN);
1174            out.q_reduced.push(Number::NAN);
1175            out.q_sign.push(0);
1176            out.sigma.push(0.0);
1177            continue;
1178        };
1179        let d = kinv[next];
1180        next += 1;
1181        // natural units, as `compute` exports them: the scaled row
1182        // multiplier carries df/dg and the scaled slack dg
1183        let dg = d_scale.map_or(1.0, |v| v[row]);
1184        let sigma = sigma_s[row] * dg * dg / obj_scale;
1185        // `a = ã ⊙ d` leaves the row's own `dg` in, and the reported
1186        // curvature is natural, so it comes back out here
1187        let norm2 = norm2[slot_of[row].expect("every solved row has a norm slot")] / (dg * dg);
1188        if norm2 <= 0.0 {
1189            // no direction to measure curvature along, exactly as in
1190            // the report -- and the geometric weight is degenerate at
1191            // zero gradient, so the raw `Σ/floor` lower bound stands
1192            let e = zero_gradient_row(sigma * sgn, floor);
1193            out.status.push(e.status);
1194            out.ratio.push(e.ratio);
1195            out.q_reduced.push(Number::NAN);
1196            out.q_sign.push(0);
1197            out.sigma.push(sigma);
1198            continue;
1199        }
1200        if !d.is_finite() {
1201            out.status.push(UNIDENTIFIED);
1202            out.ratio.push(Number::NAN);
1203            out.q_reduced.push(Number::NAN);
1204            out.q_sign.push(0);
1205            out.sigma.push(sigma);
1206            continue;
1207        }
1208        // `(K⁻¹)_{ss} = 0` -- the rest of the model determines the
1209        // row's value outright, so there is no direction left to
1210        // reduce along -- sends `q` to an infinity the ratio divides
1211        // to zero, i.e. INACTIVE: whatever holds the row there, it is
1212        // not its own bound. A `(K⁻¹)_{ss}` at roundoff level lands in
1213        // the same class from either side of zero, which is why no
1214        // guard branches on its sign; a genuinely negative reduced
1215        // curvature is modest in magnitude and reports through
1216        // `q_sign`, exactly as an indefinite `∇dᵀH∇d` does in the
1217        // report.
1218        let q = (1.0 / d - sigma) * norm2;
1219        // the geometric weight against the curvature along the unit
1220        // normal: `compute`'s pairing, so a decoupled row's ratio here
1221        // IS the report's
1222        let e = classify_entry(sigma * norm2 * sgn, q * sgn, floor, mu);
1223        let bounded = rhas_l[row] || rhas_u[row];
1224        out.status.push(if bounded { e.status } else { UNBOUNDED });
1225        out.ratio.push(if bounded { e.ratio } else { Number::NAN });
1226        out.q_reduced.push(q);
1227        out.q_sign.push(sign_of(q));
1228        out.sigma.push(sigma);
1229    }
1230    debug_assert_eq!(next, kinv.len(), "every inequality row consumed its solve");
1231    Ok(out)
1232}
1233
1234#[cfg(test)]
1235mod tests {
1236    use super::*;
1237    use crate::backsolver::SensBacksolver;
1238    use crate::barrier::{BarrierIterate, BoundSide, KktOperator};
1239    use pounce_linalg::triplet::{GenTMatrix, GenTMatrixSpace, SymTMatrix, SymTMatrixSpace};
1240    use std::rc::Rc;
1241
1242    const MU: Number = 1e-6;
1243
1244    /// An engine that supplies the classification's data and nothing else.
1245    /// Two variables and two inequality rows, each with a lower bound at a
1246    /// unit slack. The first bound's multiplier puts `s·z` at `μ`, on the
1247    /// central path, and the second's at `100μ`, off it.
1248    struct TwoBoundsPerSide;
1249
1250    fn side() -> BoundSide {
1251        BoundSide {
1252            pos: vec![0, 1],
1253            mult: vec![MU, 100.0 * MU],
1254            slack: vec![1.0, 1.0],
1255        }
1256    }
1257
1258    impl SensBacksolver for TwoBoundsPerSide {
1259        fn dim(&self) -> usize {
1260            self.block_dims().iter().sum()
1261        }
1262        fn solve(&self, _rhs: &[Number], _lhs: &mut [Number]) -> bool {
1263            false
1264        }
1265    }
1266
1267    impl BarrierIterate for TwoBoundsPerSide {
1268        fn block_dims(&self) -> [usize; 8] {
1269            [2, 2, 0, 2, 2, 0, 2, 0]
1270        }
1271        fn barrier_mu(&self) -> Number {
1272            MU
1273        }
1274        fn converged_iterate(&self, _out: &mut [Number]) -> Result<(), String> {
1275            Err("not used by the classification".into())
1276        }
1277        fn barrier_residual(&self, _point: &[Number], _out: &mut [Number]) -> Result<(), String> {
1278            Err("not used by the classification".into())
1279        }
1280        fn corrector_operator<'a>(
1281            &'a self,
1282            _predicted: &[Number],
1283            _pinned: &[(usize, Number)],
1284            _exact_hessian: bool,
1285        ) -> Result<Box<dyn KktOperator + 'a>, String> {
1286            Err("not used by the classification".into())
1287        }
1288        fn activity_data(&self) -> Result<ActivityData, String> {
1289            let mut hess = SymTMatrix::new(SymTMatrixSpace::new(2, vec![1, 2], vec![1, 2]));
1290            hess.set_values(&[1.0, 1.0]);
1291            let mut jac = GenTMatrix::new(GenTMatrixSpace::new(2, 2, vec![1, 2], vec![1, 2]));
1292            jac.set_values(&[1.0, 1.0]);
1293            Ok(ActivityData {
1294                x_l: side(),
1295                x_u: BoundSide::default(),
1296                d_l: side(),
1297                d_u: BoundSide::default(),
1298                sigma_x: vec![MU, 100.0 * MU],
1299                sigma_s: vec![MU, 100.0 * MU],
1300                hessian: Rc::new(hess),
1301                jac_c: Rc::new(GenTMatrix::new(GenTMatrixSpace::new(0, 2, vec![], vec![]))),
1302                jac_d: Rc::new(jac),
1303            })
1304        }
1305        fn model_maps(&self) -> Result<ModelMaps, String> {
1306            Ok(ModelMaps {
1307                n_full_x: 2,
1308                var_x_to_full_x: vec![0, 1],
1309                full_g_to_c: vec![None, None],
1310                full_g_to_d: vec![Some(0), Some(1)],
1311            })
1312        }
1313        fn scaling(&self) -> EngineScaling {
1314            EngineScaling::none()
1315        }
1316        fn primal_bounds(&self) -> Result<(Vec<Number>, Vec<Number>), String> {
1317            Err("not used by the classification".into())
1318        }
1319    }
1320
1321    /// Each bound's central-path check reads its own multiplier and slack,
1322    /// not those of another bound on the same side.
1323    #[test]
1324    fn each_bound_is_checked_against_the_central_path_with_its_own_product() {
1325        let report = compute(&TwoBoundsPerSide).expect("the classification");
1326        assert_eq!(report.var_off_central_path, vec![false, true]);
1327        assert_eq!(report.row_off_central_path, vec![false, true]);
1328    }
1329}