Skip to main content

Module continuation_path

Module continuation_path 

Source
Expand description

Object 1 — the ContinuationPath: one object that couples the three annealing schedules that today live separately and uncoupled, so a K≥2 SAE joint fit always arrives via a regime where the inner problem is a contraction — never solved cold.

§The three schedules, coupled along one scalar path parameter s

Three homotopy legs presently advance on their own clocks:

  1. ρ-anneal — heavy oversmoothing penalty ρ₀ ≫ ρ* down to the target ρ*. Owned by the spine [crate::estimate::reml::continuation::fit_with_continuation] (the callable ρ-anneal primitive, promoted from a private warm-start helper). At large ρ the penalized Hessian dominates the likelihood Hessian, so the inner P-IRLS / arrow-Schur solve is strongly convex — a contraction.
  2. Assignment temperature τ — diffuse softmax / IBP relaxation (high τ) sharpened toward the near-discrete MAP active set (low τ). Owned by gam_sae::manifold::GumbelTemperatureSchedule. High τ makes the assignment map smooth and far from the combinatorial argmax cliff.
  3. Isometry weight — loose analytic isometry gauge (small w) ramped to the tight target weight. Owned by gam_terms::analytic_penalties::ScalarWeightSchedule on gam_terms::analytic_penalties::IsometryPenalty. A loose gauge leaves the decoder free to find a good fit before the gauge pins it.

ContinuationPath advances all three in lockstep along a single scalar path parameter s ∈ [1 → 0]. s = 1 is the entry regime: large ρ, high τ, loose-but-rising isometry — the regime in which the joint inner solve is provably a contraction. s = 0 is the real objective: target ρ*, sharp τ, tight isometry. The path walks s monotonically down, advancing the underlying schedules at each waypoint so the inner problem is never asked to jump from cold to the real objective.

§Entry is always the heavy-smoothing regime

There is no “solve cold at the real objective” entry. The only entry is s = 1, where every leg is at its smoothing extreme and the inner solve is a contraction. A K≥2 SAE joint fit therefore always arrives at ρ* / τ_min / tight-isometry along a continuous descent from a regime where convergence is guaranteed.

§The tail is a homotopy FLOOR, not a gate

If a downward step’s inner solve struggles, the path does not reject: it re-enters a heavier regime (raises s back toward 1 by a back-off fraction) and re-descends with a finer step. This is a floor the iterate bounces off, never a trapdoor it falls through. The structural guarantee is encoded in the type: [ContinuationStep] — the per-step outcome enum — has no Reject / Failed / NoUsableSeed arm. The worst a step can report is [ContinuationStep::Reentered] (bounced off the floor, re-descending), which is progress toward, not abandonment of, the fit. There is no value of the outcome type that means “give up”.

§How this absorbs #969 (warm-invariance) and #976 (hardening)

  • #969 — warm-invariance. A cold entry (no warm β) and a warm entry (β carried from a previous fit / cache) both enter at s = 1, where the inner solve is a contraction with a unique fixed point. A contraction forgets its initial condition, so both entries are funneled to the SAME s = 1 iterate, and from there walk the SAME coupled schedule to the SAME criterion at s = 0. Warm entry only shortens the walk (its β is already near the s = 1 fixed point); it cannot change the destination. The path therefore makes “cold and warm reach the same criterion” a structural property rather than a tolerance the caller must check.
  • #976 — hardening. Two hooks the wiring agent (editing rho_optimizer.rs / atom_selection.rs) calls per inner iteration: a trust-region cap on the assignment logits (LogitTrustRegion) so a single Newton step can never fling the relaxed assignment across the argmax cliff; and an active-mass-floor breach signal (ActiveMassFloor / MassFloorBreach) that, when the per-row active mass collapses toward the uniform saddle, triggers a re-seed from the scaffold (the pristine seeded geometry) — recorded in the ReseedLedger, never fatal. A breach is a ledger entry and a regime re-entry, not an error return.

This module owns the coupling object and the hook interfaces / return types. The wiring agent implements the call sites against these types.

Structs§

ActiveMassFloor
Active-mass-floor watcher (#976). The wiring agent calls ActiveMassFloor::check with the per-row mean active assignment mass each inner iteration. When the mass collapses toward the uniform saddle (below the floor), check returns a MassFloorBreach the caller records in the ReseedLedger and acts on by re-seeding from the scaffold. A breach is never fatal — there is no error return.
ContinuationPath
Object 1 — the coupled continuation path. Owns the three schedules and the scalar path parameter s, and drives the K≥2 SAE joint fit down the coupled homotopy. Entry is always s = 1 (heavy-smoothing contraction regime); the tail is a homotopy floor with no rejection exit.
CoupledSchedules
The coupled schedule state that ContinuationPath owns. Each leg is the concrete schedule object the rest of the codebase already advances; the path holds them so they can only ever move together.
LegEndpoints
The endpoints of one coupled annealing leg, in path-parameter terms. at_entry is the value at s = 1 (heavy-smoothing regime); at_target is the value at s = 0 (real objective). Interpolation is in the leg’s own natural geometry (log-space for ρ and τ, linear-in-weight for the isometry gauge, matching each schedule’s current_* law).
LogitTrustRegion
Per-iteration trust-region cap on the assignment logits.
MassFloorBreach
A recorded active-mass-floor breach. Carries the observed mass and the floor it fell below. The wiring agent’s response is a re-seed-from-scaffold, not a failure: this is appended to the ReseedLedger and the path re-enters a heavier regime.
ReseedEvent
One scaffold re-seed event: the path parameter s at which the breach was observed and the breach payload. Lets diagnostics see whether re-seeds cluster at sharp-τ waypoints (the expected near-cliff regime).
ReseedLedger
Append-only ledger of scaffold re-seeds triggered by active-mass-floor breaches. Non-fatal by construction: the ledger only records; it never holds a terminal/abort state. The wiring agent threads one ledger through the joint fit and queries ReseedLedger::reseed_count for diagnostics.
ScalarLegTargets
The lockstep target values of the two scalar legs at a given s. Handed to the wiring agent so it can install τ on the SAE term and the isometry weight on the gauge penalty before the spine pass at this waypoint.

Enums§

LogitStepCap
Outcome of applying the logit trust-region cap to a proposed step. The wiring agent applies the returned (possibly shrunk) step. There is no “reject” outcome — the cap only scales the step.
PathDemotionReason
Why the seed cascade asked the path to demote a seed to a heavier regime. Purely a diagnostic tag carried into the demotion ledger — every variant resolves to “re-enter the same seed at a heavier s”, never to a rejection.
PathRegime
The coarse “which heavy-smoothing regime is the path currently entering at” view the seed cascade reports against. Banded from the live path parameter s ∈ [0, 1]: heavier regime ⇒ larger s ⇒ deeper into the contraction basin. Every variant is a re-entry the cascade re-evaluates a seed at; none of them is a rejection.

Constants§

CONTINUATION_WALK_BUDGET
Total leg budget for one coupled walk (rho_optimizer.rs drives ContinuationPath::step at most this many times per seed). Two legs per waypoint: a clean walk uses CONTINUATION_WAYPOINTS descents, and each homotopy-floor bounce costs ~2 extra legs at the one-notch REENTRY_BACKOFF (the re-entry leg plus one finer re-descent leg), so a 2× budget tolerates ~CONTINUATION_WAYPOINTS/2 bounces before the walk is cut off — enough for the expected near-cliff re-entries while keeping the total inner-solve count bounded. The previous 1.5× budget tolerated only ~1 two-notch bounce, so any mass-floor bounce ended the walk un-arrived.
CONTINUATION_WAYPOINTS
Number of lockstep waypoints the path visits as s walks 1 → 0. Each waypoint advances every leg one notch and runs one ρ-anneal spine pass. Chosen so the geometric schedules have room to descend an order of magnitude or two per leg without a single step that crosses the contraction boundary; the homotopy floor absorbs any waypoint that still over-reaches.
REENTRY_BACKOFF
Back-off fraction applied to s when a waypoint’s inner solve struggles: s ← min(1, s + REENTRY_BACKOFF). Re-entering a heavier regime and re-descending with a halved step is the floor behavior — there is no rejection alternative.
S_STEP_FLOOR
Floor on the per-waypoint descent step in s. Below this the path is taking near-zero steps; it does not give up — it pins s at its current (heavier) regime and keeps re-descending from there. The floor is a behavior, never an exit.
WALK_EVAL_CEILING
Hard ceiling on budgeted spine evals across one coupled walk — the #968 termination guarantee made structural. A clean walk budgets PATH_BUDGET + (CONTINUATION_WAYPOINTS − 1) · WARM_LEG_EVAL_BUDGET (one cold entry spine, then warm legs); the ceiling leaves ~3× that for homotopy-floor bounces. At the ceiling the path arrives with its best converged state instead of spending another leg: a walk cannot spin.
WARM_LEG_EVAL_BUDGET
Eval budget for one warm waypoint leg. A warm leg starts at the previous waypoint’s converged state and walks one waypoint of ρ, so it needs a handful of evals, not the full cold spine: the coupled path’s waypoints ARE the anneal. (Re-running the whole ρ₀→target spine per waypoint multiplies the walk’s cost by the spine budget — the K=2 existence fixture burned 7 CPU-hours exactly that way before warm legs existed.)

Functions§

couple_schedules
Convenience: build the standard coupled schedules for a K≥2 SAE joint fit from the ρ box and the τ / isometry schedules the term already carries.
mean_active_mass
View helper: the wiring agent passes the SAE assignment matrix (rows × atoms) to compute the mean active mass for the ActiveMassFloor check. Defined here so the floor’s input convention has one owner.