Skip to main content

pounce_algorithm/
second_opinion.rs

1//! The second-opinion ladder: re-solving a failure along a different
2//! trajectory before believing it.
3//!
4//! A local-infeasibility verdict on a nonconvex problem is a *local*
5//! statement, and an `Invalid_Number_Detected` is a statement about the
6//! callbacks at one point. Both are frequently artifacts of the trajectory
7//! the solve happened to take, or of the point it started from, rather than
8//! facts about the model. The ladder re-solves along up to three deliberately
9//! different trajectories and promotes a re-solve only if it converges.
10//!
11//! This module holds the **policy** — which rungs apply to which failure, what
12//! each rung changes, and how a ladder's outcome resolves. It reads options
13//! and returns descriptions; it runs no solves. The **driver** that applies a
14//! rung and calls back into the solver lives in `pounce-restoration`, because
15//! each rung has to rebuild the restoration sub-IPM's factory provider and
16//! that provider is defined one crate up the dependency graph.
17//!
18//! Split out of `pounce-cli`'s `main.rs` so the Python, C and Rust embedding
19//! surfaces get the same ladder rather than each re-deriving it — the
20//! asymmetry mattered, because a caller driving POUNCE from a modelling layer
21//! is precisely the one most likely to hand over an uninitialized (and so
22//! all-zero, and so possibly rank-deficient) starting point.
23
24use pounce_common::options_list::OptionsList;
25use pounce_nlp::SolveStatistics;
26use pounce_nlp::return_codes::ApplicationReturnStatus;
27
28/// One rung of the local-infeasibility second-opinion ladder: a label for the
29/// console plus the option assignments that define this re-solve's trajectory.
30///
31/// Assignments are applied on top of the *baseline* options, not on top of the
32/// previous rung — see `second_opinion_rungs`.
33#[derive(Debug, Clone, PartialEq, Eq)]
34pub struct SecondOpinionRung {
35    pub label: &'static str,
36    /// The knob this rung varies, and *only* that knob — one
37    /// `read_from_str` line per assignment.
38    ///
39    /// A rung does not carry lines undoing the earlier rungs. It used to,
40    /// and that was a defect: the undo was written as the baseline's
41    /// *resolved* value, so a knob the caller never set came back **set**.
42    /// `mu_strategy` is the one that bites — `is_mu_strategy_fallback_enabled`
43    /// is default-on only while `mu_strategy` is unset, so rung 3 writing
44    /// back a resolved `monotone` silently switched off pounce's own
45    /// μ-strategy stall retry for the duration of the rung. Measured on
46    /// KRONOS `a18_ackley1`: the displaced solve stalls at `max_iter`, the
47    /// flip that would have certified it in 237 iterations never fires, and
48    /// the ladder reports no recovery. Restoring by *value* is not the same
49    /// as restoring by *set-ness*.
50    ///
51    /// The driver restores the baseline with
52    /// `OptionSnapshot::apply` before every rung, which does honour
53    /// set-ness, so a rung starts from the true baseline by construction.
54    pub assignments: Vec<String>,
55}
56
57/// Which failure opened the ladder. Not every rung is evidence about every
58/// failure: an `Invalid_Number_Detected` is a statement about the *callbacks*
59/// at a point, and re-running the same callbacks at the same point under a
60/// different linear-solver scaling or a different barrier strategy evaluates
61/// the same non-finite quantity again. Only the rung that moves the point
62/// applies there.
63#[derive(Debug, Clone, Copy, PartialEq, Eq)]
64pub enum SecondOpinionTrigger {
65    /// `Infeasible_Problem_Detected` — a *local* statement about a nonconvex
66    /// problem, which every rung is evidence against.
67    LocalInfeasibility,
68    /// `Invalid_Number_Detected` — a NaN or infinity out of the model.
69    InvalidNumber,
70    /// `Restoration_Failed` — the restoration phase could not find a point
71    /// the filter would accept. Like the two above and unlike a budget exit,
72    /// this is a statement about the trajectory the solve happened to take,
73    /// not about the model; see [`SecondOpinionTrigger::for_status`].
74    RestorationFailure,
75    /// `Restoration_Failed` from one exit only: gh#1042's unreleased
76    /// hand-back, where restoration converged to a point feasible for the
77    /// original problem that it never released, handed it back, and the
78    /// outer re-entered restoration from that very point
79    /// (`SolveStatistics::unreleased_handback_exit`). Every rung
80    /// [`Self::RestorationFailure`] opens, plus `mu_strategy=adaptive` ahead
81    /// of them.
82    ///
83    /// The extra rung is measured, not reasoned. Before the exit reported
84    /// upstream's status it reported `Error_In_Step_Computation`, which
85    /// opens the adaptive rung alone, and on arm64 that rung recovered
86    /// Vanderbei `brainpc0` / `brainpc1` / `brainpc2` (2117 → 956 base,
87    /// 1073 → 378, 561 → 258, each then solved by adaptive), while
88    /// `brainpc0` under `start_point_perturbation` ran to 3000 iterations
89    /// and failed. Opening it on every `Restoration_Failed` would instead
90    /// change the ladder of the whole gh#815 family, where the displaced
91    /// start is the measured winner (13 of 15 against adaptive's 4), so the
92    /// trigger is this exit's alone. It is produced only by
93    /// [`Self::for_solve`]; [`Self::for_status`] never returns it, because
94    /// the status alone cannot tell the two `Restoration_Failed`s apart.
95    UnreleasedHandback,
96    /// `Error_In_Step_Computation` — the step computation broke down: the
97    /// KKT system could not be made to yield a usable step, even after
98    /// regularization. Like a restoration failure and unlike a budget exit,
99    /// this is a statement about the trajectory the solve happened to take,
100    /// so a different barrier schedule is evidence against it.
101    ///
102    /// Measured on `deb7` (found via gh#960): the wasm32 build, whose factorization
103    /// rounds differently from the native one, follows the native trajectory
104    /// for 83 iterations, diverges by one ulp in `inf_du` at iteration 84,
105    /// and ends `Error_In_Step_Computation` at 160 where native solves in
106    /// 147. Under `mu_strategy=adaptive` the same binary and the same model
107    /// reach the same optimum as native — `Solve_Succeeded`, 131 iterations,
108    /// objective 97.559938 — so the rescue was already in the ladder and
109    /// only the trigger was missing. This opens **only** that rung: on the
110    /// same case `feral_scaling=mc64` still fails at 154 iterations,
111    /// `feral_increase_quality=no` at 201, and `start_point_perturbation`
112    /// spends 324 iterations to fail anyway.
113    StepComputationError,
114    /// `Maximum_Iterations_Exceeded` — **and only when the solve escalated
115    /// the linear solver's factorization quality at least once** (gh#857).
116    ///
117    /// This is the one trigger that is not a property of the verdict alone,
118    /// and the exception it carves out of the paragraph on
119    /// [`SecondOpinionTrigger::for_status`] is narrow on purpose. A budget
120    /// exit normally wants a bigger budget, not a different trajectory —
121    /// but a `feral_increase_quality` escalation *changes which pivots are
122    /// taken and never steps back down*, so when one fired, the wall the
123    /// solve hit may be the escalated trajectory's rather than the model's,
124    /// and a bigger budget re-runs the same wall. On
125    /// `square_flowsheet_resto`'s lbfgs leg the escalated path reaches 3000
126    /// iterations and the un-escalated one converges in 178.
127    ///
128    /// The escalation count is what keeps this from opening a ladder on
129    /// every budget exit; it comes from the `quality_escalations` statistic
130    /// and is checked in [`second_opinion_rungs`], not here — `for_status`
131    /// only sees a status. A solve that escalated zero times produces an
132    /// empty rung list and the driver returns before it narrates anything.
133    IterationLimit,
134}
135
136/// What the baseline options already provide, so a rung that would be a no-op
137/// can be dropped instead of burning a solve to re-derive the same answer.
138#[derive(Debug, Clone, Copy)]
139pub struct SecondOpinionAvailability {
140    pub trigger: SecondOpinionTrigger,
141    pub scaling_retry_enabled: bool,
142    pub mu_retry_enabled: bool,
143    pub perturbed_start_retry_enabled: bool,
144    pub already_mc64: bool,
145    pub already_adaptive: bool,
146    /// The baseline already displaces the start, so there is no displacement
147    /// left for the third rung to add that the failing solve did not have.
148    pub already_perturbed: bool,
149    /// `feral_increase_quality_retry` (gh#857), rung 4's own enable.
150    pub increase_quality_retry_enabled: bool,
151    /// The baseline already ran with `feral_increase_quality=no`, so rung 4
152    /// would re-run the solve that just failed.
153    pub already_no_increase_quality: bool,
154    /// How many times the failing solve's linear solver actually accepted an
155    /// `increase_quality` escalation — the `quality_escalations` statistic.
156    ///
157    /// The only entry here that is a *measurement of the solve* rather than
158    /// a reading of its options, and rung 4 is unimplementable without it:
159    /// an escalation moves no field a report carries, so "did this solve
160    /// take the rung that has a documented losing direction" cannot be
161    /// answered from the verdict. `0` means provably not a candidate, and
162    /// rung 4 is dropped — which is what stops a budget exit on a
163    /// never-escalating model from paying for an extra solve.
164    ///
165    /// It is a gate at `>= 1`, deliberately **not** a threshold. `deb7` and
166    /// `square_flowsheet_resto`'s base solve escalate exactly twice each on
167    /// their exact legs, one gaining the solve and one losing it, so no
168    /// count separates them; the verdict does.
169    pub baseline_quality_escalations: u64,
170    /// `infeasibility_saddle_escape_retry`, rung 5's own enable.
171    pub saddle_escape_retry_enabled: bool,
172    /// The failing solve's point is **not** a local minimizer of the
173    /// max-norm violation: [`crate::infeasibility_saddle::infeasibility_saddle_escape`]
174    /// found a nearby point with less of it.
175    ///
176    /// Like `baseline_quality_escalations` this is a measurement of the
177    /// solve, not a reading of its options, and it is `false` until the
178    /// driver has run the probe. The probe needs the verdict's point, which
179    /// only the application holds after the solve, so it cannot be computed
180    /// in [`SecondOpinionAvailability::from_options`].
181    pub baseline_saddle_escape_found: bool,
182    /// `feral_scaling` tag naming the baseline's *resolved* scaling strategy.
183    /// `None` under `ScalingStrategy::External`, which drops rungs 2 and 3.
184    ///
185    /// Now purely a gate. It was the tag those rungs wrote back to undo rung
186    /// 1, and `None` meant there was nothing to write; the driver's snapshot
187    /// restores by set-ness instead, so no tag is needed and the External
188    /// case would in fact be safe to run. The gate is kept because dropping
189    /// it adds two rungs on externally-scaled models — a trajectory change,
190    /// and a separate decision from this bug fix.
191    pub baseline_scaling: Option<&'static str>,
192}
193
194/// Build the ladder of second-opinion re-solves for a failing verdict, in the
195/// order they should be tried. Each rung varies exactly one knob from the
196/// baseline options.
197///
198/// 1. **`feral_scaling=mc64` — numerical diversity**
199///    (`feral_infeasibility_scaling_retry`, on by default). Some KKT
200///    trajectories are chaotic: under two equally backward-stable
201///    linear-solver scalings the iterates stay bit-identical for many
202///    iterations, then diverge by ~1 ULP and fall into different basins — one
203///    optimal, the other a spurious stationary point of the constraint
204///    violation (`discs.nl`: InfNorm → infeasible, MC64/Identity/MA57/IPOPT →
205///    optimal). Sensitive dependence, not a bad solve, so the a-priori
206///    scaling router cannot tell the two apart and no per-factor residual
207///    flags it; the only reliable signal is the whole-solve verdict.
208///
209/// 2. **`mu_strategy=adaptive` — algorithmic diversity**
210///    (`infeasibility_mu_strategy_retry`, on by default). Rung 1 perturbs only
211///    the linear algebra, so it is evidence *only* when the trajectory is
212///    ULP-hypersensitive. When it is not, MC64 retraces the same iterates and
213///    agrees for the same reason the first solve was wrong — on gh #524
214///    (`cresc4`, 6 vars / 8 constraints, feasible, Ipopt solves it in 71
215///    iterations) the MC64 re-solve reproduced the original trajectory
216///    bit-identically and "corroborated" the false verdict. A different
217///    barrier strategy changes the iterate sequence itself, which is what the
218///    monotone-µ default gets wrong here: adaptive µ walks to the known
219///    optimum. This is also the remedy IPOPT's own documentation gives a user
220///    who gets an infeasibility verdict on a problem they believe is feasible;
221///    running it automatically just spares them the round trip.
222///
223/// 3. **`start_point_perturbation=1e-2` — a different starting point**
224///    (`infeasibility_perturbed_start_retry`, on by default). The only rung
225///    that moves the point rather than the path, and so the only one that is
226///    evidence about an `Invalid_Number_Detected` or a `Restoration_Failed`
227///    — see [`SecondOpinionTrigger`]. Those two triggers open this rung and
228///    nothing else, so they cost exactly one extra solve.
229///
230/// 4. **`feral_increase_quality=no` — undo the factorization escalation**
231///    (`feral_increase_quality_retry`, on by default; gh#857). The only rung
232///    whose gate is a *measurement of the failing solve* rather than a
233///    reading of its options: it opens on a `Restoration_Failed`, a
234///    `Maximum_Iterations_Exceeded` or an `Infeasible_Problem_Detected`
235///    **and** only when the solve's `quality_escalations` count is at
236///    least 1.
237///
238///    The infeasibility trigger was added after the other two, and the
239///    reason is worth keeping: `square_flowsheet_resto`'s lbfgs leg does
240///    **not** exit the same way on every platform. On macOS/arm64 it runs
241///    to the 3000-iteration cap and exits `Maximum_Iterations_Exceeded`;
242///    on linux/x86_64 the same 3000 iterations with the same 25
243///    escalations end `Infeasible_Problem_Detected` instead — a *wrong
244///    answer* on a feasible model, and one the three infeasibility rungs
245///    above do not recover. The verdict an escalation-rerouted trajectory
246///    produces is not a property of the escalation, so pinning the rung to
247///    two of the three shapes left the fix firing on one platform and not
248///    the other.
249///
250///    Unlike the other two triggers this one is **not** free, and the
251///    difference is worth stating: a `Restoration_Failed` or a budget exit
252///    is a failure either way, so the rung's cost lands only on runs that
253///    were already going to report one, whereas a *genuine* infeasibility
254///    verdict is a correct answer and the rung can only confirm it. Six
255///    fixture-legs pay exactly that — one extra solve, e.g.
256///    `issue_508_infeasible_gap_1em4` 982 → 1423 total iterations, with no
257///    status, objective, iteration count or engine moving. The `>= 1` gate
258///    is what bounds it: of the eight NLP-arm infeasibility fixture-legs,
259///    four escalated and take the rung and four are untouched.
260///    `feral_increase_quality` is
261///    on by default and genuinely two-sided — it buys accuracy and 15–25% of
262///    the iterations on several fixture-legs and loses whole solves on others
263///    — and its losing direction previously had no automatic recovery at all.
264///    Measured on `square_flowsheet_resto` under feral 0.20: the lbfgs leg
265///    escalates 25 times, hits the 3000-iteration cap, and converges in 178
266///    with the rung off. Under feral 0.21 that leg hits the cap with the rung
267///    off as well (the `square_system_lm` stage recovers it), and no corpus
268///    fixture reaches this rung at default options; the tests' witness is the
269///    exact leg at `max_iter=75` (base MaxIter after 3 escalations, this rung
270///    Optimal at 59). The other 178s and 131s in this file are 0.20 figures.
271///
272///    It is **appended**, not inserted, so a `Restoration_Failed` that rung 3
273///    already recovers (the gh#815 family, and this same fixture's exact leg)
274///    reaches promotion first and costs nothing new. The `>= 1` is a gate and
275///    not a threshold: `deb7` escalates exactly as many times as
276///    `square_flowsheet_resto`'s base solve and *gains* by it, so a count
277///    cannot separate the two — only the verdict can, and `deb7`'s is
278///    `Optimal`.
279///
280///    Opening this rung also stands the µ-strategy stall retry down
281///    (`Application::run_with_mu_strategy_fallback`), which is what keeps it
282///    from being a third solve. That retry fires unconditionally on
283///    `Maximum_Iterations_Exceeded`, so before gh#857 this fixture's lbfgs leg
284///    paid 3000 capped iterations, then a second full 3000 under the flipped
285///    schedule that escalated 25 times again and ended no better, and only
286///    then reached this rung's 178. The flip is blind and the escalation is
287///    measured; skipping it takes the run from three solves to two, changing
288///    no reported number — which is also why the fixture sweep is
289///    byte-identical across that change.
290///
291/// 5. **`restart_from_infeasibility_saddle` — restart beside the verdict's
292///    point** (`infeasibility_saddle_escape_retry`, on by default). Local
293///    infeasibility only, and only when
294///    [`crate::infeasibility_saddle::infeasibility_saddle_escape`] finds a
295///    point near the verdict's point with less max-norm violation, which
296///    shows the verdict's point is not a local minimizer of the violation.
297///    The solve restarts from that point. Rungs 1–4 vary how the solve gets
298///    somewhere; this one starts the solve from the other side of where it
299///    got stuck, which is the only thing that helps when the end point is a
300///    saddle every trajectory is attracted to. Measured on `discs` under
301///    `feral_scaling=mc64`: discs 8 and 9 end on the same centre, the
302///    tangency row between them reads `0 = 4` with an exactly zero gradient,
303///    and rungs 2–4 all end at the same kind of point.
304///
305/// Rungs are **not** cumulative: the driver restores the baseline before each
306/// rung, so rung 2 runs without rung 1's scaling and rung 3 without either
307/// earlier knob. That reset is load-bearing, not tidiness: on gh #524's
308/// `cresc4`, `mu_strategy=adaptive` recovers the optimum but
309/// `mu_strategy=adaptive` with `feral_scaling=mc64` still reports local
310/// infeasibility, so a cumulative ladder would have discarded the fix.
311pub fn second_opinion_rungs(avail: SecondOpinionAvailability) -> Vec<SecondOpinionRung> {
312    let mut rungs = Vec::new();
313    let infeasible = avail.trigger == SecondOpinionTrigger::LocalInfeasibility;
314    if infeasible && avail.scaling_retry_enabled && !avail.already_mc64 {
315        rungs.push(SecondOpinionRung {
316            label: "feral_scaling=mc64",
317            assignments: vec!["feral_scaling mc64\n".to_string()],
318        });
319    }
320    // The barrier-schedule rung is the one rung a step-computation
321    // breakdown opens, and the only one measured to recover one: the
322    // trajectory walked into a KKT system no regularization could make
323    // usable, and a different μ schedule is a different trajectory. See
324    // `SecondOpinionTrigger::StepComputationError` for the deb7 numbers.
325    if avail.baseline_scaling.is_some()
326        && (infeasible
327            || matches!(
328                avail.trigger,
329                SecondOpinionTrigger::StepComputationError
330                    | SecondOpinionTrigger::UnreleasedHandback
331            ))
332        && avail.mu_retry_enabled
333        && !avail.already_adaptive
334    {
335        rungs.push(SecondOpinionRung {
336            label: "mu_strategy=adaptive",
337            assignments: vec!["mu_strategy adaptive\n".to_string()],
338        });
339    }
340    // Rung 3 varies exactly one knob from the *baseline*, so both earlier
341    // rungs' knobs must be undone first, not inherited — gh #524 is the case
342    // where stacking two of them threw the fix away. That undo is the
343    // driver's job (`OptionSnapshot::apply` before each rung), not an
344    // assignment here; see the note on `assignments`.
345    //
346    // Not on the iteration-limit trigger. That trigger is gated on a
347    // measurement (gh#857, rung 4 below) and exists to test one hypothesis —
348    // that the escalation is what walked the solve into the wall. Displacing
349    // the start of a solve that ran out of budget tests nothing: it starts a
350    // fresh trajectory with the same budget and the same escalating ladder
351    // waiting for it. Opening it here would put an extra solve on every
352    // escalating budget exit for no reason anyone has measured.
353    // Not on the step-computation trigger either, and for a measured
354    // reason rather than the reasoning above: on deb7 it spends 324
355    // iterations and fails anyway, where the rung above solves in 131.
356    if avail.baseline_scaling.is_some()
357        && !matches!(
358            avail.trigger,
359            SecondOpinionTrigger::IterationLimit | SecondOpinionTrigger::StepComputationError
360        )
361        && avail.perturbed_start_retry_enabled
362        && !avail.already_perturbed
363    {
364        rungs.push(SecondOpinionRung {
365            label: "start_point_perturbation=1e-2",
366            assignments: vec!["start_point_perturbation 1e-2\n".to_string()],
367        });
368    }
369    // 4. `feral_increase_quality=no` — undo the escalation (gh#857).
370    //
371    // Appended, never prepended. On a `Restoration_Failed` the rung above
372    // already recovers the gh#815 family *and* `square_flowsheet_resto`'s own
373    // exact leg, and it promotes and breaks before this one runs, so those
374    // solves cost nothing new. This rung is what is left when that fails, and
375    // it is the whole ladder on an `Maximum_Iterations_Exceeded`.
376    //
377    // The `>= 1` is the gate the trigger's doc describes: a solve that never
378    // escalated cannot have been rerouted by an escalation, so there is
379    // nothing here to test and no solve to spend.
380    if matches!(
381        avail.trigger,
382        SecondOpinionTrigger::RestorationFailure
383            | SecondOpinionTrigger::UnreleasedHandback
384            | SecondOpinionTrigger::IterationLimit
385            | SecondOpinionTrigger::LocalInfeasibility
386    ) && avail.increase_quality_retry_enabled
387        && !avail.already_no_increase_quality
388        && avail.baseline_quality_escalations >= 1
389    {
390        rungs.push(SecondOpinionRung {
391            label: "feral_increase_quality=no",
392            assignments: vec!["feral_increase_quality no\n".to_string()],
393        });
394    }
395    // 5. Restart from beside an infeasibility saddle.
396    //
397    // Local infeasibility only, and only when the probe refuted the verdict's
398    // claim that its point minimizes the violation. It carries no option
399    // assignment: the driver starts this solve from the probe's point instead
400    // of the model's. Appended last, so every ladder that a rung above
401    // already settles runs exactly as before; on a genuinely infeasible model
402    // whose verdict point is a strict minimizer the probe finds nothing and
403    // this rung never exists.
404    if infeasible && avail.saddle_escape_retry_enabled && avail.baseline_saddle_escape_found {
405        rungs.push(SecondOpinionRung {
406            label: crate::infeasibility_saddle::SADDLE_ESCAPE_RUNG_LABEL,
407            assignments: Vec::new(),
408        });
409    }
410    rungs
411}
412
413/// Whether the ladder's narration should reach the console.
414///
415/// `print_level 0` is a request for silence, and the ladder's running
416/// commentary is no more exempt from it than the `EXIT:` block — the C
417/// interface is an Ipopt drop-in, where `print_level=0 sb=yes` is the
418/// documented way to get a quiet solve, and up to five unexpected `pounce:`
419/// lines on a failing one is exactly what that asks not to happen. The ladder
420/// still *runs*; only the console is quiet.
421///
422/// Lives here rather than at each call site because the CLI and the C
423/// interface both need it and a duplicated branch is a branch that can drift
424/// — one of them silently losing the gate would look identical to the other
425/// keeping it. `pounce-rs` and Python do not call this: they never print,
426/// they hand the narration back to the caller.
427///
428/// Only an *explicit* `print_level` silences: a level nobody set narrates,
429/// and so does an unregistered or unreadable one. That is the pre-gate
430/// behaviour and the safe direction — too much on a failing solve, never too
431/// little.
432pub fn narration_is_wanted(options: &OptionsList) -> bool {
433    options
434        .get_integer_value("print_level", "")
435        .map(|(level, found)| !found || level >= 1)
436        .unwrap_or(true)
437}
438
439/// Did a second-opinion re-solve converge well enough to overturn the original
440/// local-infeasibility verdict? Only a clean or acceptable-level solve
441/// promotes; everything else (including a second infeasibility verdict) leaves
442/// the original verdict standing.
443pub fn scaling_retry_promoted(retry_status: ApplicationReturnStatus) -> bool {
444    matches!(
445        retry_status,
446        ApplicationReturnStatus::SolveSucceeded | ApplicationReturnStatus::SolvedToAcceptableLevel
447    )
448}
449
450/// Resolve the final `(status, statistics)` after an MC64 hypersensitivity
451/// re-solve (code review L23).
452///
453/// On promotion the retry is the authoritative solve, so its status **and** its
454/// statistics are reported together. Otherwise the original local-infeasibility
455/// verdict is kept — and so are the *original* solve's statistics, so the
456/// summary / JSON report never pair the original verdict with the failed
457/// retry's iteration count or objective. The pre-fix code reverted `status` to
458/// `InfeasibleProblemDetected` but read `app.statistics()` *after* the retry,
459/// leaking the retry solve's stats into a report labeled with the original
460/// verdict.
461pub fn resolve_scaling_retry_outcome(
462    original_status: ApplicationReturnStatus,
463    retry_status: ApplicationReturnStatus,
464    original_stats: SolveStatistics,
465    retry_stats: SolveStatistics,
466) -> (ApplicationReturnStatus, SolveStatistics) {
467    if scaling_retry_promoted(retry_status) {
468        (retry_status, retry_stats)
469    } else {
470        (original_status, original_stats)
471    }
472}
473
474impl SecondOpinionTrigger {
475    /// Which ladder, if any, a finished solve's verdict opens.
476    ///
477    /// Only these three statuses open one. In particular an iteration- or
478    /// time-limit exit does not: the answer there is a bigger budget, and a
479    /// re-solve from a different trajectory would burn the same budget again
480    /// to reach the same wall.
481    ///
482    /// `Restoration_Failed` is on the list for the same reason the other two
483    /// are (gh#815): it is a report about the *path*, not about the model.
484    /// The restoration phase failing to find a filter-acceptable point says
485    /// the iterate reached somewhere the sub-problem could not work from, and
486    /// a different starting point is a different sub-problem. It is not a
487    /// budget exit — pounce stops far short of `max_iter` — so "give it more
488    /// iterations" is not the available answer, which is precisely the
489    /// distinction the paragraph above draws. Measured on the gh#815 square
490    /// flowsheet family: both failing members exit `Restoration_Failed`, no
491    /// ladder ran, and rung 3 alone recovers both to `Optimal Solution
492    /// Found` — one of them (`f100`) to an optimum Ipopt itself misses.
493    ///
494    /// Only rung 3 opens on this trigger; rungs 1 and 2 stay gated on
495    /// [`SecondOpinionTrigger::LocalInfeasibility`], so a restoration failure
496    /// costs exactly one extra solve. (One exit's `Restoration_Failed` is the
497    /// exception, and the status cannot show it: see
498    /// [`SecondOpinionTrigger::UnreleasedHandback`] and [`Self::for_solve`].) That is the measured ordering, not
499    /// caution for its own sake: over the KRONOS corpus the displaced start
500    /// recovered 13 of 15 where `mu_strategy=adaptive` recovered 4 (see
501    /// `start_point_retry`'s option text).
502    pub fn for_status(status: ApplicationReturnStatus) -> Option<Self> {
503        match status {
504            ApplicationReturnStatus::InfeasibleProblemDetected => {
505                Some(SecondOpinionTrigger::LocalInfeasibility)
506            }
507            ApplicationReturnStatus::InvalidNumberDetected => {
508                Some(SecondOpinionTrigger::InvalidNumber)
509            }
510            ApplicationReturnStatus::RestorationFailed => {
511                Some(SecondOpinionTrigger::RestorationFailure)
512            }
513            // The exception to the paragraph above, and it is an exception
514            // to the *reason* rather than a change of mind about it
515            // (gh#857). "A re-solve would burn the same budget to reach the
516            // same wall" is true of a budget exit whose trajectory the
517            // ladder cannot change — and a `feral_increase_quality`
518            // escalation is precisely a trajectory the ladder *can* change,
519            // because it persists across every later factorization and one
520            // option removes it. Measured: `square_flowsheet_resto`'s lbfgs
521            // leg hits the 3000-iteration cap having escalated 25 times, and
522            // converges in 178 with the rung off.
523            //
524            // This returns `Some` for every budget exit; the escalation
525            // count is not visible here. `second_opinion_rungs` drops the
526            // rung when the count is zero, so a non-escalating budget exit
527            // opens no *option* rung — `a_budget_exit_that_never_escalated_opens_no_rung`
528            // is the pin. It can still open the ladder's last stage: on a
529            // model `square_system::probe` admits (square, equality-only,
530            // `∇f` exactly zero at the start) the driver runs the
531            // square-system LM on any trigger this returns, escalation or
532            // not, because that stage replaces the method rather than
533            // re-running its trajectory. On every other model a
534            // non-escalating budget exit is unchanged — the driver returns
535            // before narrating, with the same status, statistics and
536            // console.
537            ApplicationReturnStatus::MaximumIterationsExceeded => {
538                Some(SecondOpinionTrigger::IterationLimit)
539            }
540            ApplicationReturnStatus::ErrorInStepComputation => {
541                Some(SecondOpinionTrigger::StepComputationError)
542            }
543            _ => None,
544        }
545    }
546
547    /// [`Self::for_status`], refined by what the finished solve reports about
548    /// how it got there. The one refinement: a `Restoration_Failed` that the
549    /// gh#1042 exit produced is [`Self::UnreleasedHandback`], which opens the
550    /// `mu_strategy=adaptive` rung no other restoration failure opens.
551    pub fn for_solve(
552        status: ApplicationReturnStatus,
553        statistics: &SolveStatistics,
554    ) -> Option<Self> {
555        match Self::for_status(status) {
556            Some(SecondOpinionTrigger::RestorationFailure)
557                if statistics.unreleased_handback_exit =>
558            {
559                Some(SecondOpinionTrigger::UnreleasedHandback)
560            }
561            other => other,
562        }
563    }
564
565    /// The verdict alone, with no claim about how the solve got there —
566    /// for a console line that does not depend on an escalation having
567    /// happened (the square-system stage opens on a budget exit that never
568    /// escalated, where [`Self::describe`]'s wording would be false).
569    pub fn verdict(self) -> &'static str {
570        match self {
571            SecondOpinionTrigger::IterationLimit => "iteration limit",
572            other => other.describe(),
573        }
574    }
575
576    /// The word for this trigger in the option rungs' console line. For
577    /// [`SecondOpinionTrigger::IterationLimit`] it names the escalation,
578    /// which is true there: no option rung opens on a budget exit that did
579    /// not escalate.
580    pub fn describe(self) -> &'static str {
581        match self {
582            SecondOpinionTrigger::LocalInfeasibility => "local infeasibility",
583            SecondOpinionTrigger::InvalidNumber => "invalid number",
584            SecondOpinionTrigger::RestorationFailure => "restoration failure",
585            SecondOpinionTrigger::UnreleasedHandback => {
586                "restoration failure at an unreleased feasible hand-back"
587            }
588            SecondOpinionTrigger::StepComputationError => "step computation breakdown",
589            SecondOpinionTrigger::IterationLimit => {
590                "iteration limit after a factorization escalation"
591            }
592        }
593    }
594}
595
596impl SecondOpinionAvailability {
597    /// Read everything the ladder needs to know about the baseline solve out
598    /// of the options it ran under.
599    ///
600    /// The scaling and barrier tags are read from the **resolved** strategy,
601    /// not from the option strings. `feral_scaling` is applied only when set
602    /// explicitly and otherwise `FeralConfig::from_env()` governs via
603    /// `POUNCE_FERAL_SCALING`, so the option string reads `auto` for an
604    /// env-configured run; writing that back would silently override the
605    /// environment on the retry instead of restoring it. `External` is
606    /// unreachable from the string option, and if it ever arrives here there
607    /// is no tag to write, so the rungs that need one are dropped rather than
608    /// guessed at.
609    /// `baseline_quality_escalations` is the failing solve's
610    /// `SolveStatistics::quality_escalations`, and it is a **required
611    /// parameter** rather than a defaulted setter for the reason gh#857
612    /// exists at all: the quantity is invisible everywhere else, so a caller
613    /// that forgot it would silently pass `0`, rung 4 would never open, and
614    /// the recovery would look like a rung that simply does not fire. A
615    /// missing argument is a compile error; a forgotten setter is a
616    /// regression nobody can see.
617    pub fn from_options(
618        options: &OptionsList,
619        trigger: SecondOpinionTrigger,
620        baseline_quality_escalations: u64,
621    ) -> Self {
622        // Each of the three `*_retry` flags is read with its tag written out
623        // as a literal rather than through a shared closure: `init_options_wiring`
624        // proves every registered Initialization option is actually consumed by
625        // scanning the source for `get_*_value("<tag>"`, and a closure taking
626        // the tag as a parameter is invisible to that scan — which would leave
627        // `infeasibility_perturbed_start_retry` looking like a knob that
628        // validates, accepts a value and does nothing.
629        let scaling = crate::application::feral_config_from_options(options).scaling;
630        let baseline_scaling = match scaling {
631            pounce_feral::ScalingStrategy::Auto => Some("auto"),
632            pounce_feral::ScalingStrategy::InfNorm => Some("infnorm"),
633            pounce_feral::ScalingStrategy::Mc64Symmetric => Some("mc64"),
634            pounce_feral::ScalingStrategy::Identity => Some("identity"),
635            pounce_feral::ScalingStrategy::External(_) => None,
636        };
637        // The *resolved* strategy, not the raw option: under a
638        // limited-memory Hessian an unset `mu_strategy` already runs
639        // adaptive (gh#746), and the rung would re-run the base solve.
640        let already_adaptive = crate::application::mu_strategy_resolves_to_adaptive(options);
641        Self {
642            trigger,
643            scaling_retry_enabled: options
644                .get_bool_value("feral_infeasibility_scaling_retry", "")
645                .map(|(v, _found)| v)
646                .unwrap_or(true),
647            mu_retry_enabled: options
648                .get_bool_value("infeasibility_mu_strategy_retry", "")
649                .map(|(v, _found)| v)
650                .unwrap_or(true),
651            perturbed_start_retry_enabled: options
652                .get_bool_value("infeasibility_perturbed_start_retry", "")
653                .map(|(v, _found)| v)
654                .unwrap_or(true),
655            already_mc64: matches!(scaling, pounce_feral::ScalingStrategy::Mc64Symmetric),
656            already_adaptive,
657            already_perturbed: options
658                .get_numeric_value("start_point_perturbation", "")
659                .map(|(v, _found)| v > 0.0)
660                .unwrap_or(false),
661            increase_quality_retry_enabled: options
662                .get_bool_value("feral_increase_quality_retry", "")
663                .map(|(v, _found)| v)
664                .unwrap_or(true),
665            // The rung would set `feral_increase_quality` to the value the
666            // failing solve already ran under, so it is a no-op re-solve.
667            // Read as a *value*, not as set-ness: the default is `yes`, so
668            // an unset option is not already-no.
669            already_no_increase_quality: options
670                .get_bool_value("feral_increase_quality", "")
671                .map(|(v, _found)| !v)
672                .unwrap_or(false),
673            baseline_quality_escalations,
674            saddle_escape_retry_enabled: options
675                .get_bool_value("infeasibility_saddle_escape_retry", "")
676                .map(|(v, _found)| v)
677                .unwrap_or(true),
678            baseline_saddle_escape_found: false,
679            baseline_scaling,
680            // `mu_strategy` has exactly two registered values, so "not
681            // adaptive" is "monotone" and there is no third case to guess at.
682        }
683    }
684}
685
686#[cfg(test)]
687mod scaling_retry_tests {
688    use super::{
689        SecondOpinionAvailability, SecondOpinionTrigger, narration_is_wanted,
690        resolve_scaling_retry_outcome, scaling_retry_promoted, second_opinion_rungs,
691    };
692    use pounce_common::options_list::OptionsList;
693    use pounce_nlp::SolveStatistics;
694    use pounce_nlp::return_codes::ApplicationReturnStatus;
695
696    fn avail() -> SecondOpinionAvailability {
697        SecondOpinionAvailability {
698            trigger: SecondOpinionTrigger::LocalInfeasibility,
699            scaling_retry_enabled: true,
700            mu_retry_enabled: true,
701            perturbed_start_retry_enabled: true,
702            already_mc64: false,
703            already_adaptive: false,
704            already_perturbed: false,
705            increase_quality_retry_enabled: true,
706            already_no_increase_quality: false,
707            // Zero, so that every test written before gh#857 keeps asserting
708            // exactly the ladder it asserted then: rung 4's gate is a count,
709            // and a solve that did not escalate cannot reach it. The tests
710            // that are about rung 4 raise it explicitly, which is also how
711            // they document that the count is the thing doing the work.
712            baseline_quality_escalations: 0,
713            saddle_escape_retry_enabled: true,
714            // False for the same reason the escalation count is zero: rung 5
715            // is gated on a measurement, and the tests written before it keep
716            // asserting the ladders they asserted then.
717            baseline_saddle_escape_found: false,
718            baseline_scaling: Some("auto"),
719        }
720    }
721
722    /// The default ladder is three rungs, in increasing order of how much
723    /// they change: linear algebra, then barrier trajectory, then the point
724    /// the trajectory starts from.
725    #[test]
726    fn default_ladder_is_scaling_then_barrier_strategy_then_start() {
727        let rungs = second_opinion_rungs(avail());
728        let labels: Vec<_> = rungs.iter().map(|r| r.label).collect();
729        assert_eq!(
730            labels,
731            [
732                "feral_scaling=mc64",
733                "mu_strategy=adaptive",
734                "start_point_perturbation=1e-2"
735            ]
736        );
737    }
738
739    /// gh #524: the rungs are applied to the *baseline*, not stacked, because
740    /// on `cresc4` `mu_strategy=adaptive` recovers the optimum while
741    /// `mu_strategy=adaptive` together with `feral_scaling=mc64` still reports
742    /// local infeasibility — a cumulative ladder would throw the fix away.
743    ///
744    /// The barrier rung therefore carries `mu_strategy` and nothing else;
745    /// rung 1's scaling is undone by the driver re-applying its snapshot, and
746    /// `second_opinion_driver::tests::each_rung_starts_from_the_baseline`
747    /// is where the undo itself is pinned.
748    #[test]
749    fn barrier_rung_varies_only_the_barrier_strategy() {
750        for baseline in ["auto", "infnorm"] {
751            let rungs = second_opinion_rungs(SecondOpinionAvailability {
752                baseline_scaling: Some(baseline),
753                ..avail()
754            });
755            let barrier = rungs
756                .iter()
757                .find(|r| r.label == "mu_strategy=adaptive")
758                .expect("barrier rung present");
759            let assigned: Vec<_> = barrier.assignments.iter().map(|a| a.trim()).collect();
760            assert_eq!(assigned, ["mu_strategy adaptive"]);
761        }
762    }
763
764    /// A rung that cannot change anything is dropped rather than burning a
765    /// whole solve to re-derive the same answer.
766    #[test]
767    fn rungs_already_satisfied_at_baseline_are_dropped() {
768        let only_barrier = second_opinion_rungs(SecondOpinionAvailability {
769            already_mc64: true,
770            ..avail()
771        });
772        assert_eq!(
773            only_barrier.iter().map(|r| r.label).collect::<Vec<_>>(),
774            ["mu_strategy=adaptive", "start_point_perturbation=1e-2"],
775        );
776
777        let only_scaling = second_opinion_rungs(SecondOpinionAvailability {
778            already_adaptive: true,
779            ..avail()
780        });
781        assert_eq!(
782            only_scaling.iter().map(|r| r.label).collect::<Vec<_>>(),
783            ["feral_scaling=mc64", "start_point_perturbation=1e-2"],
784        );
785
786        assert!(
787            second_opinion_rungs(SecondOpinionAvailability {
788                already_mc64: true,
789                already_adaptive: true,
790                already_perturbed: true,
791                ..avail()
792            })
793            .is_empty(),
794            "nothing left to vary means no ladder at all",
795        );
796    }
797
798    /// A resolved scaling with no `feral_scaling` tag
799    /// (`ScalingStrategy::External`) drops the barrier rung; the scaling rung
800    /// is unaffected. The gate is now conservatism rather than necessity —
801    /// the driver restores by set-ness, so a missing tag no longer strands
802    /// the rung under rung 1's scaling — and it is kept because lifting it
803    /// would add rungs on externally-scaled models, which is a trajectory
804    /// change. See the note on `baseline_scaling`.
805    #[test]
806    fn barrier_rung_is_dropped_when_the_baseline_scaling_has_no_tag() {
807        let rungs = second_opinion_rungs(SecondOpinionAvailability {
808            baseline_scaling: None,
809            ..avail()
810        });
811        assert_eq!(
812            rungs.iter().map(|r| r.label).collect::<Vec<_>>(),
813            ["feral_scaling=mc64"],
814        );
815    }
816
817    /// Each rung has its own opt-out, and turning both off restores upstream
818    /// IPOPT's behaviour of shipping the first verdict.
819    #[test]
820    fn each_rung_can_be_disabled_independently() {
821        assert_eq!(
822            second_opinion_rungs(SecondOpinionAvailability {
823                scaling_retry_enabled: false,
824                ..avail()
825            })
826            .iter()
827            .map(|r| r.label)
828            .collect::<Vec<_>>(),
829            ["mu_strategy=adaptive", "start_point_perturbation=1e-2"],
830        );
831        assert_eq!(
832            second_opinion_rungs(SecondOpinionAvailability {
833                mu_retry_enabled: false,
834                ..avail()
835            })
836            .iter()
837            .map(|r| r.label)
838            .collect::<Vec<_>>(),
839            ["feral_scaling=mc64", "start_point_perturbation=1e-2"],
840        );
841        assert_eq!(
842            second_opinion_rungs(SecondOpinionAvailability {
843                perturbed_start_retry_enabled: false,
844                ..avail()
845            })
846            .iter()
847            .map(|r| r.label)
848            .collect::<Vec<_>>(),
849            ["feral_scaling=mc64", "mu_strategy=adaptive"],
850        );
851        assert!(
852            second_opinion_rungs(SecondOpinionAvailability {
853                scaling_retry_enabled: false,
854                mu_retry_enabled: false,
855                perturbed_start_retry_enabled: false,
856                ..avail()
857            })
858            .is_empty(),
859        );
860    }
861
862    /// gh #524's lesson applied to the third rung: it varies exactly one thing
863    /// from the *baseline*. It undoes the earlier rungs by having the driver
864    /// re-apply the snapshot, so its own assignment list is the displacement
865    /// and nothing else.
866    #[test]
867    fn start_rung_assigns_only_the_displacement() {
868        for baseline_scaling in ["auto", "infnorm"] {
869            let rungs = second_opinion_rungs(SecondOpinionAvailability {
870                baseline_scaling: Some(baseline_scaling),
871                ..avail()
872            });
873            let start = rungs
874                .iter()
875                .find(|r| r.label == "start_point_perturbation=1e-2")
876                .expect("start rung present");
877            let assigned: Vec<_> = start.assignments.iter().map(|a| a.trim()).collect();
878            assert_eq!(assigned, ["start_point_perturbation 1e-2"]);
879        }
880    }
881
882    /// The regression this file exists to prevent a second time.
883    ///
884    /// Rungs 2 and 3 used to undo their predecessors by writing the
885    /// baseline's *resolved* value back. For a knob the caller never set that
886    /// is a no-op by value and a change by set-ness, and
887    /// `is_mu_strategy_fallback_enabled` reads set-ness: it is default-on
888    /// only while `mu_strategy` is unset. So rung 3 re-asserting a resolved
889    /// `monotone` turned pounce's own μ-strategy stall retry off for the
890    /// length of the rung. On KRONOS `a18_ackley1` that is the difference
891    /// between `Solve_Succeeded` in 237 iterations and
892    /// `Maximum_Iterations_Exceeded` at 3000.
893    ///
894    /// No rung may name a knob it is not there to vary — restoring is the
895    /// driver's job, because only the snapshot knows set-ness.
896    #[test]
897    fn no_rung_writes_back_a_knob_it_does_not_vary() {
898        for avail in [
899            avail(),
900            SecondOpinionAvailability {
901                already_adaptive: true,
902                ..avail()
903            },
904            SecondOpinionAvailability {
905                trigger: SecondOpinionTrigger::InvalidNumber,
906                ..avail()
907            },
908        ] {
909            for rung in second_opinion_rungs(avail) {
910                let varies = rung.label.split('=').next().expect("label has a tag");
911                for a in &rung.assignments {
912                    let tag = a.trim().split_whitespace().next().expect("tag");
913                    assert_eq!(
914                        tag,
915                        varies,
916                        "rung `{}` writes `{}`, which it does not vary",
917                        rung.label,
918                        a.trim(),
919                    );
920                }
921            }
922        }
923    }
924
925    /// Rung 3 sits behind the same `baseline_scaling` gate as rung 2 and is
926    /// dropped with it, for the same reason: kept as conservatism about a
927    /// trajectory change on externally-scaled models, not because the rung
928    /// needs a tag to put back.
929    #[test]
930    fn start_rung_is_dropped_when_the_baseline_scaling_has_no_tag() {
931        let rungs = second_opinion_rungs(SecondOpinionAvailability {
932            baseline_scaling: None,
933            ..avail()
934        });
935        assert!(
936            !rungs
937                .iter()
938                .any(|r| r.label == "start_point_perturbation=1e-2"),
939            "{:?}",
940            rungs.iter().map(|r| r.label).collect::<Vec<_>>(),
941        );
942    }
943
944    /// gh#1042 review: under a limited-memory Hessian an unset `mu_strategy`
945    /// already resolves to adaptive (gh#746), so the `mu_strategy=adaptive`
946    /// rung would re-run the base solve byte-for-byte. Measured on `deb7`
947    /// L-BFGS before this: base and "adaptive" rung both 1610 iterations,
948    /// `f = 99.636943584`. The ladder must read the resolved strategy.
949    #[test]
950    fn the_adaptive_rung_is_skipped_when_limited_memory_already_runs_adaptive() {
951        let labels = |opts: &OptionsList| {
952            let a = SecondOpinionAvailability::from_options(
953                opts,
954                SecondOpinionTrigger::StepComputationError,
955                0,
956            );
957            second_opinion_rungs(a)
958                .iter()
959                .map(|r| r.label)
960                .collect::<Vec<_>>()
961        };
962        let mut lbfgs = OptionsList::new();
963        lbfgs
964            .set_string_value("hessian_approximation", "limited-memory", true, true)
965            .unwrap();
966        assert!(
967            !labels(&lbfgs).contains(&"mu_strategy=adaptive"),
968            "limited-memory with mu_strategy unset already runs adaptive: {:?}",
969            labels(&lbfgs)
970        );
971        // An explicit monotone under limited-memory is a real alternative.
972        let mut pinned = lbfgs.clone();
973        pinned
974            .set_string_value("mu_strategy", "monotone", true, true)
975            .unwrap();
976        assert!(labels(&pinned).contains(&"mu_strategy=adaptive"));
977        // The exact arm, unset: monotone, so the rung is a different solve.
978        let exact = OptionsList::new();
979        assert!(labels(&exact).contains(&"mu_strategy=adaptive"));
980    }
981
982    /// The console gate, which both printing surfaces share. An explicit `0`
983    /// is silence; anything above it, or no setting at all, stays loud — a
984    /// caller who did not ask for quiet is not asking for less.
985    #[test]
986    fn narration_follows_print_level() {
987        let mut opts = OptionsList::new();
988        // Unset: narrate, the pre-gate behaviour.
989        assert!(narration_is_wanted(&opts));
990        // …and an unset level narrates whatever the registered default reads
991        // as, which is what distinguishes "nobody asked" from "asked for 0".
992        for (level, want) in [(0, false), (1, true), (5, true), (12, true)] {
993            opts.set_integer_value("print_level", level, true, true)
994                .unwrap();
995            assert_eq!(narration_is_wanted(&opts), want, "print_level={level}");
996        }
997    }
998
999    /// An `Invalid_Number_Detected` reaches only the rung that moves the
1000    /// point. Re-running the same callbacks at the same point under a
1001    /// different linear-solver scaling or a different barrier strategy
1002    /// evaluates the same non-finite quantity again, so those two rungs are
1003    /// not evidence about this failure and would only burn solves.
1004    #[test]
1005    fn an_invalid_number_reaches_only_the_start_rung() {
1006        let rungs = second_opinion_rungs(SecondOpinionAvailability {
1007            trigger: SecondOpinionTrigger::InvalidNumber,
1008            ..avail()
1009        });
1010        assert_eq!(
1011            rungs.iter().map(|r| r.label).collect::<Vec<_>>(),
1012            ["start_point_perturbation=1e-2"],
1013        );
1014    }
1015
1016    /// gh#815. A restoration failure opens the ladder, and opens exactly the
1017    /// one rung that is evidence about it. Rungs 1 and 2 vary the *path* from
1018    /// the same starting point; the restoration sub-problem failed because of
1019    /// where the iterate got to, and a different path can arrive somewhere
1020    /// just as bad. Rung 3 moves the point, which makes it a different
1021    /// sub-problem — and it is the rung the KRONOS measurement ranks first
1022    /// (13 of 15 against `mu_strategy=adaptive`'s 4).
1023    #[test]
1024    fn a_restoration_failure_reaches_only_the_start_rung() {
1025        let rungs = second_opinion_rungs(SecondOpinionAvailability {
1026            trigger: SecondOpinionTrigger::RestorationFailure,
1027            ..avail()
1028        });
1029        assert_eq!(
1030            rungs.iter().map(|r| r.label).collect::<Vec<_>>(),
1031            ["start_point_perturbation=1e-2"],
1032        );
1033    }
1034
1035    /// gh#857 rung 4 on a restoration failure, and the ordering claim that
1036    /// makes it cheap: it is **appended**, so the gh#815 rung still runs
1037    /// first and still promotes first. `square_flowsheet_resto`'s exact leg
1038    /// is exactly that case, and is why this rung costs it nothing.
1039    #[test]
1040    fn an_escalating_restoration_failure_appends_the_quality_rung() {
1041        let rungs = second_opinion_rungs(SecondOpinionAvailability {
1042            trigger: SecondOpinionTrigger::RestorationFailure,
1043            baseline_quality_escalations: 2,
1044            ..avail()
1045        });
1046        assert_eq!(
1047            rungs.iter().map(|r| r.label).collect::<Vec<_>>(),
1048            ["start_point_perturbation=1e-2", "feral_increase_quality=no",],
1049        );
1050    }
1051
1052    /// gh#1042. The one `Restoration_Failed` that also opens the barrier
1053    /// rung: the unreleased-hand-back exit. Adaptive goes first — it is what
1054    /// recovered Vanderbei `brainpc0`/`1`/`2` on arm64 when this exit still
1055    /// reported `Error_In_Step_Computation`, where the displaced start ran
1056    /// `brainpc0` to 3000 iterations and failed — and every rung a plain
1057    /// restoration failure opens follows it, in the same order.
1058    #[test]
1059    fn an_unreleased_handback_exit_opens_the_barrier_rung_first() {
1060        let rungs = second_opinion_rungs(SecondOpinionAvailability {
1061            trigger: SecondOpinionTrigger::UnreleasedHandback,
1062            ..avail()
1063        });
1064        assert_eq!(
1065            rungs.iter().map(|r| r.label).collect::<Vec<_>>(),
1066            ["mu_strategy=adaptive", "start_point_perturbation=1e-2"],
1067        );
1068        let rungs = second_opinion_rungs(SecondOpinionAvailability {
1069            trigger: SecondOpinionTrigger::UnreleasedHandback,
1070            baseline_quality_escalations: 2,
1071            ..avail()
1072        });
1073        assert_eq!(
1074            rungs.iter().map(|r| r.label).collect::<Vec<_>>(),
1075            [
1076                "mu_strategy=adaptive",
1077                "start_point_perturbation=1e-2",
1078                "feral_increase_quality=no",
1079            ],
1080        );
1081        // Under L-BFGS an unset strategy is already adaptive (gh#746), so
1082        // the rung would repeat the base solve; the trigger does not
1083        // override that skip.
1084        let rungs = second_opinion_rungs(SecondOpinionAvailability {
1085            trigger: SecondOpinionTrigger::UnreleasedHandback,
1086            already_adaptive: true,
1087            ..avail()
1088        });
1089        assert_eq!(
1090            rungs.iter().map(|r| r.label).collect::<Vec<_>>(),
1091            ["start_point_perturbation=1e-2"],
1092        );
1093    }
1094
1095    /// The refinement is the exit's alone: `for_solve` names
1096    /// `UnreleasedHandback` only for a `Restoration_Failed` whose statistics
1097    /// carry the exit's flag, and changes nothing else — not a plain
1098    /// restoration failure, and not another status that happens to carry
1099    /// the flag (the exit's `Infeasible_Problem_Detected` arm, or an
1100    /// acceptable-point rollback).
1101    #[test]
1102    fn only_the_unreleased_handback_exit_refines_a_restoration_failure() {
1103        use ApplicationReturnStatus as A;
1104        let flagged = SolveStatistics {
1105            unreleased_handback_exit: true,
1106            ..SolveStatistics::default()
1107        };
1108        let plain = SolveStatistics::default();
1109        assert_eq!(
1110            SecondOpinionTrigger::for_solve(A::RestorationFailed, &flagged),
1111            Some(SecondOpinionTrigger::UnreleasedHandback),
1112        );
1113        assert_eq!(
1114            SecondOpinionTrigger::for_solve(A::RestorationFailed, &plain),
1115            Some(SecondOpinionTrigger::RestorationFailure),
1116        );
1117        for status in [
1118            A::InfeasibleProblemDetected,
1119            A::SolvedToAcceptableLevel,
1120            A::ErrorInStepComputation,
1121            A::MaximumIterationsExceeded,
1122            A::InvalidNumberDetected,
1123        ] {
1124            assert_eq!(
1125                SecondOpinionTrigger::for_solve(status, &flagged),
1126                SecondOpinionTrigger::for_status(status),
1127                "{status:?}: the flag must not change this verdict's ladder"
1128            );
1129        }
1130    }
1131
1132    /// The whole ladder on a budget exit is rung 4, and it is there only
1133    /// because the solve escalated.
1134    #[test]
1135    fn an_escalating_budget_exit_reaches_only_the_quality_rung() {
1136        let rungs = second_opinion_rungs(SecondOpinionAvailability {
1137            trigger: SecondOpinionTrigger::IterationLimit,
1138            baseline_quality_escalations: 25,
1139            ..avail()
1140        });
1141        assert_eq!(
1142            rungs.iter().map(|r| r.label).collect::<Vec<_>>(),
1143            ["feral_increase_quality=no"],
1144        );
1145        assert_eq!(rungs[0].assignments, ["feral_increase_quality no\n"]);
1146    }
1147
1148    /// The third shape an escalation-rerouted trajectory takes, and the one
1149    /// that is platform-dependent.
1150    ///
1151    /// `square_flowsheet_resto`'s lbfgs leg runs 3000 iterations and escalates
1152    /// 25 times on both macOS/arm64 and linux/x86_64, and then exits
1153    /// `Maximum_Iterations_Exceeded` on the first and
1154    /// `Infeasible_Problem_Detected` on the second — a wrong answer on a
1155    /// feasible model that the un-escalated solve reaches in 178 iterations.
1156    /// Rung 4 opened on two of the three shapes until that divergence turned
1157    /// up in CI, which meant the gh#857 fix recovered the model on one
1158    /// platform and not the other.
1159    ///
1160    /// Appended here too, so the three infeasibility rungs still run and still
1161    /// promote first: this is what is left when they do not.
1162    #[test]
1163    fn an_escalating_infeasibility_verdict_appends_the_quality_rung() {
1164        let rungs = second_opinion_rungs(SecondOpinionAvailability {
1165            trigger: SecondOpinionTrigger::LocalInfeasibility,
1166            baseline_quality_escalations: 25,
1167            ..avail()
1168        });
1169        assert_eq!(
1170            rungs.iter().map(|r| r.label).collect::<Vec<_>>(),
1171            [
1172                "feral_scaling=mc64",
1173                "mu_strategy=adaptive",
1174                "start_point_perturbation=1e-2",
1175                "feral_increase_quality=no",
1176            ],
1177        );
1178    }
1179
1180    /// And the gate holds on that trigger too: a local-infeasibility verdict
1181    /// from a solve that never escalated opens the same three rungs it opened
1182    /// before gh#857, and pays for no fourth.
1183    #[test]
1184    fn an_infeasibility_verdict_that_never_escalated_gets_no_quality_rung() {
1185        let rungs = second_opinion_rungs(SecondOpinionAvailability {
1186            trigger: SecondOpinionTrigger::LocalInfeasibility,
1187            baseline_quality_escalations: 0,
1188            ..avail()
1189        });
1190        assert_eq!(
1191            rungs.iter().map(|r| r.label).collect::<Vec<_>>(),
1192            [
1193                "feral_scaling=mc64",
1194                "mu_strategy=adaptive",
1195                "start_point_perturbation=1e-2",
1196            ],
1197        );
1198    }
1199
1200    /// The other branch of rung 4's gate, and the one that keeps `for_status`
1201    /// naming a trigger for every budget exit from costing anything. Without
1202    /// this the change would put an extra solve on every
1203    /// `Maximum_Iterations_Exceeded` in the corpus.
1204    ///
1205    /// An empty ladder is not merely "no rung ran": the driver returns
1206    /// `unchanged` on an empty list *before* it narrates, so such a solve is
1207    /// byte-identical to its pre-gh#857 self.
1208    #[test]
1209    fn a_budget_exit_that_never_escalated_opens_no_rung() {
1210        let rungs = second_opinion_rungs(SecondOpinionAvailability {
1211            trigger: SecondOpinionTrigger::IterationLimit,
1212            baseline_quality_escalations: 0,
1213            ..avail()
1214        });
1215        assert!(
1216            rungs.is_empty(),
1217            "a budget exit with no escalation has nothing for the ladder to \
1218             test, and must not pay for a solve: {:?}",
1219            rungs.iter().map(|r| r.label).collect::<Vec<_>>(),
1220        );
1221    }
1222
1223    /// Rung 5 opens only on a local-infeasibility verdict whose point the
1224    /// probe showed is not a minimizer of the violation, and it comes last.
1225    #[test]
1226    fn the_saddle_rung_needs_the_probe_and_comes_last() {
1227        let label = crate::infeasibility_saddle::SADDLE_ESCAPE_RUNG_LABEL;
1228        let labels = |a: SecondOpinionAvailability| -> Vec<&'static str> {
1229            second_opinion_rungs(a).iter().map(|r| r.label).collect()
1230        };
1231        let found = SecondOpinionAvailability {
1232            baseline_saddle_escape_found: true,
1233            ..avail()
1234        };
1235        assert_eq!(labels(found).last(), Some(&label));
1236        assert!(
1237            second_opinion_rungs(found)
1238                .last()
1239                .unwrap()
1240                .assignments
1241                .is_empty()
1242        );
1243        // Not found: the ladder is exactly what it was before rung 5.
1244        assert!(!labels(avail()).contains(&label));
1245        // Switched off.
1246        assert!(
1247            !labels(SecondOpinionAvailability {
1248                saddle_escape_retry_enabled: false,
1249                ..found
1250            })
1251            .contains(&label)
1252        );
1253        // Only local infeasibility: the probe is about that verdict's claim.
1254        for trigger in [
1255            SecondOpinionTrigger::InvalidNumber,
1256            SecondOpinionTrigger::RestorationFailure,
1257            SecondOpinionTrigger::StepComputationError,
1258            SecondOpinionTrigger::IterationLimit,
1259        ] {
1260            assert!(
1261                !labels(SecondOpinionAvailability {
1262                    trigger,
1263                    baseline_quality_escalations: 3,
1264                    ..found
1265                })
1266                .contains(&label),
1267                "{trigger:?}"
1268            );
1269        }
1270    }
1271
1272    /// The displacement rung does **not** open on a budget exit, even an
1273    /// escalating one. It is evidence about a failed *path*, not about a
1274    /// budget, and adding it here would double the cost of the recovery
1275    /// while testing a hypothesis nobody has measured.
1276    #[test]
1277    fn a_budget_exit_does_not_reach_the_displacement_rung() {
1278        let rungs = second_opinion_rungs(SecondOpinionAvailability {
1279            trigger: SecondOpinionTrigger::IterationLimit,
1280            baseline_quality_escalations: 7,
1281            ..avail()
1282        });
1283        assert!(
1284            !rungs
1285                .iter()
1286                .any(|r| r.label == "start_point_perturbation=1e-2"),
1287            "{:?}",
1288            rungs.iter().map(|r| r.label).collect::<Vec<_>>(),
1289        );
1290    }
1291
1292    /// Rung 4 turns off with its own option, like every other rung, and is
1293    /// dropped when the baseline already ran with the escalation disabled —
1294    /// where it would re-run the solve that just failed.
1295    #[test]
1296    fn the_quality_rung_is_droppable_and_never_a_no_op() {
1297        let escalating = SecondOpinionAvailability {
1298            trigger: SecondOpinionTrigger::IterationLimit,
1299            baseline_quality_escalations: 3,
1300            ..avail()
1301        };
1302        assert!(
1303            second_opinion_rungs(SecondOpinionAvailability {
1304                increase_quality_retry_enabled: false,
1305                ..escalating
1306            })
1307            .is_empty()
1308        );
1309        assert!(
1310            second_opinion_rungs(SecondOpinionAvailability {
1311                already_no_increase_quality: true,
1312                ..escalating
1313            })
1314            .is_empty()
1315        );
1316    }
1317
1318    /// The status → trigger map is the whole opt-in surface, so pin both
1319    /// halves: the verdicts that open a ladder, and a representative exit
1320    /// that must not. `MaximumIterationsExceeded` is the case the
1321    /// doc comment argues about — a bigger budget is the answer there, and a
1322    /// re-solve would burn the same budget to reach the same wall.
1323    #[test]
1324    fn only_path_verdicts_open_a_ladder() {
1325        use ApplicationReturnStatus as A;
1326        for (status, want) in [
1327            (
1328                A::InfeasibleProblemDetected,
1329                Some(SecondOpinionTrigger::LocalInfeasibility),
1330            ),
1331            (
1332                A::InvalidNumberDetected,
1333                Some(SecondOpinionTrigger::InvalidNumber),
1334            ),
1335            (
1336                A::RestorationFailed,
1337                Some(SecondOpinionTrigger::RestorationFailure),
1338            ),
1339            // gh#857: a budget exit now names a trigger, but naming one is
1340            // not opening a ladder — the rung it names is gated on the
1341            // escalation count, which `for_status` cannot see. The pair of
1342            // tests below is what says the distinction holds.
1343            (
1344                A::MaximumIterationsExceeded,
1345                Some(SecondOpinionTrigger::IterationLimit),
1346            ),
1347            // The wasm32 case from gh#960. This row read `None` until deb7 under the wasm32
1348            // build produced a step-computation breakdown that
1349            // `mu_strategy=adaptive` recovers outright — the rescue was in
1350            // the ladder already and only the mapping was missing. Unlike
1351            // a budget exit, a breakdown is not a request for more of
1352            // anything: the trajectory reached a KKT system no
1353            // regularization could make usable, and only a different
1354            // trajectory answers it.
1355            (
1356                A::ErrorInStepComputation,
1357                Some(SecondOpinionTrigger::StepComputationError),
1358            ),
1359            (A::MaximumCpuTimeExceeded, None),
1360            (A::SolveSucceeded, None),
1361            (A::SolvedToAcceptableLevel, None),
1362        ] {
1363            assert_eq!(
1364                SecondOpinionTrigger::for_status(status),
1365                want,
1366                "{status:?} opened the wrong ladder"
1367            );
1368        }
1369    }
1370
1371    /// The wasm32 case from gh#960. A step-computation breakdown opens exactly one rung, and it
1372    /// is the barrier-schedule one.
1373    ///
1374    /// The other three are excluded on measurement, not on principle: on
1375    /// `deb7` under the wasm32 build — the case that found this — `mc64`
1376    /// still fails at 154 iterations, `feral_increase_quality=no` at 201,
1377    /// and `start_point_perturbation` burns 324 to fail anyway, while this
1378    /// rung solves in 131. A perturbed-start rung here would have doubled
1379    /// the cost of a failure it cannot fix.
1380    #[test]
1381    fn a_step_computation_breakdown_opens_only_the_barrier_rung() {
1382        let base = SecondOpinionAvailability {
1383            trigger: SecondOpinionTrigger::StepComputationError,
1384            // Rung 4's own gate is open, so only the trigger keeps it out;
1385            // otherwise this would pass for the wrong reason.
1386            baseline_quality_escalations: 2,
1387            ..avail()
1388        };
1389        assert_eq!(
1390            second_opinion_rungs(base)
1391                .iter()
1392                .map(|r| r.label)
1393                .collect::<Vec<_>>(),
1394            vec!["mu_strategy=adaptive"]
1395        );
1396
1397        // The option still governs it, and a solve already on the adaptive
1398        // schedule has nothing left to try.
1399        for off in [
1400            SecondOpinionAvailability {
1401                mu_retry_enabled: false,
1402                ..base
1403            },
1404            SecondOpinionAvailability {
1405                already_adaptive: true,
1406                ..base
1407            },
1408        ] {
1409            let rungs = second_opinion_rungs(off);
1410            assert!(
1411                rungs.is_empty(),
1412                "{:?}",
1413                rungs.iter().map(|r| r.label).collect::<Vec<_>>()
1414            );
1415        }
1416    }
1417
1418    /// …and disabling that rung leaves an invalid-number run with no ladder at
1419    /// all, rather than falling back to the two rungs that cannot help.
1420    #[test]
1421    fn an_invalid_number_with_the_start_rung_off_has_no_ladder() {
1422        assert!(
1423            second_opinion_rungs(SecondOpinionAvailability {
1424                trigger: SecondOpinionTrigger::InvalidNumber,
1425                perturbed_start_retry_enabled: false,
1426                ..avail()
1427            })
1428            .is_empty(),
1429        );
1430    }
1431
1432    /// A baseline that already displaces the start has nothing left for rung 3
1433    /// to add: re-running with the same displacement reproduces the failing
1434    /// solve.
1435    #[test]
1436    fn a_baseline_that_already_perturbs_drops_the_start_rung() {
1437        let rungs = second_opinion_rungs(SecondOpinionAvailability {
1438            already_perturbed: true,
1439            ..avail()
1440        });
1441        assert_eq!(
1442            rungs.iter().map(|r| r.label).collect::<Vec<_>>(),
1443            ["feral_scaling=mc64", "mu_strategy=adaptive"],
1444        );
1445    }
1446
1447    /// The verdict a failed ladder keeps is the one the solve actually
1448    /// shipped. Before the ladder took `Invalid_Number_Detected` as a trigger
1449    /// this function hard-coded `Infeasible_Problem_Detected`, which for the
1450    /// new trigger would have reported the wrong failure.
1451    #[test]
1452    fn a_failed_ladder_keeps_whichever_verdict_opened_it() {
1453        for original in [
1454            ApplicationReturnStatus::InfeasibleProblemDetected,
1455            ApplicationReturnStatus::InvalidNumberDetected,
1456        ] {
1457            let (status, stats) = resolve_scaling_retry_outcome(
1458                original,
1459                ApplicationReturnStatus::MaximumIterationsExceeded,
1460                stats_with_iters(7),
1461                stats_with_iters(42),
1462            );
1463            assert_eq!(status, original);
1464            assert_eq!(stats.iteration_count, 7);
1465        }
1466    }
1467
1468    fn stats_with_iters(n: i32) -> SolveStatistics {
1469        SolveStatistics {
1470            iteration_count: n,
1471            final_objective: n as f64,
1472            ..SolveStatistics::default()
1473        }
1474    }
1475
1476    /// Code review L23: when the MC64 hypersensitivity re-solve does **not**
1477    /// recover, the verdict reverts to the original local-infeasibility status
1478    /// — and the reported statistics must revert with it, not leak the failed
1479    /// retry's iteration count / objective.
1480    #[test]
1481    fn failed_retry_keeps_original_status_and_stats() {
1482        let original = stats_with_iters(7);
1483        let retry = stats_with_iters(42);
1484        for retry_status in [
1485            ApplicationReturnStatus::InfeasibleProblemDetected,
1486            ApplicationReturnStatus::MaximumIterationsExceeded,
1487            ApplicationReturnStatus::RestorationFailed,
1488        ] {
1489            assert!(!scaling_retry_promoted(retry_status));
1490            let (status, stats) = resolve_scaling_retry_outcome(
1491                ApplicationReturnStatus::InfeasibleProblemDetected,
1492                retry_status,
1493                original.clone(),
1494                retry.clone(),
1495            );
1496            assert_eq!(
1497                status,
1498                ApplicationReturnStatus::InfeasibleProblemDetected,
1499                "a non-promoting retry ({retry_status:?}) keeps the original verdict"
1500            );
1501            assert_eq!(
1502                stats.iteration_count, 7,
1503                "stats must stay the original solve's, not the failed retry's"
1504            );
1505            assert_eq!(stats.final_objective, 7.0);
1506        }
1507    }
1508
1509    /// On promotion the retry is authoritative: its status AND its statistics
1510    /// are reported together.
1511    #[test]
1512    fn promoted_retry_adopts_retry_status_and_stats() {
1513        let original = stats_with_iters(7);
1514        let retry = stats_with_iters(42);
1515        for retry_status in [
1516            ApplicationReturnStatus::SolveSucceeded,
1517            ApplicationReturnStatus::SolvedToAcceptableLevel,
1518        ] {
1519            assert!(scaling_retry_promoted(retry_status));
1520            let (status, stats) = resolve_scaling_retry_outcome(
1521                ApplicationReturnStatus::InfeasibleProblemDetected,
1522                retry_status,
1523                original.clone(),
1524                retry.clone(),
1525            );
1526            assert_eq!(status, retry_status, "a promoting retry adopts its verdict");
1527            assert_eq!(
1528                stats.iteration_count, 42,
1529                "promoted: stats must be the retry solve's"
1530            );
1531            assert_eq!(stats.final_objective, 42.0);
1532        }
1533    }
1534}