Module multinomial

Expand description

Penalized multinomial-logit (softmax) GLM driver — fixed-λ inner solve.

This is the principled vector-response companion to the scalar PIRLS path: the inner-loop Newton solver for a multi-class GAM at fixed smoothing parameters λ, using the canonical multinomial-logit likelihood (MultinomialLogitLikelihood) and the existing dense block-Fisher assembly in gam_solve::pirls::dense_block_xtwx / gam_solve::pirls::dense_block_xtwy.

§What this module does

Solve, for the reference-coded multinomial-logit GAM with K classes and design matrix X ∈ ℝ^{N×P},

    β̂ = argmin_β { − log L(β) + ½ Σ_{a=0}^{K-2} λ_a · β_a^T S β_a }

where β = [β_0; β_1; …; β_{K-2}] is the stacked coefficient vector in output-major order (β_a ∈ ℝ^P is the coefficient block for class a), S ∈ ℝ^{P×P} is the smoothing penalty matrix (shared across classes, replicated as I_{K-1} ⊗ S over the full parameter space), and λ_a is a per-class smoothing parameter.

The likelihood uses class K - 1 as the reference (η_{K-1} ≡ 0), so the softmax gauge is fixed at the η level and no additional sum-to-zero projection is required.

§Layering

Fixed-λ inner solve — fit_penalized_multinomial is the canonical coefficient-space Newton solver at given smoothing parameters λ, built on the shared crate::penalized_vector_glm engine.
REML / LAML smoothing-parameter selection — fit_penalized_multinomial_formula routes through crate::custom_family::fit_custom_family_with_rho_prior so the per-active-class λ_a are selected by the outer REML/LAML loop; the caller’s init_lambda is only a warm-start seed. The multinomial [crate::multinomial_reml::MultinomialFamily] CustomFamily impl calls the fixed-λ math above as its inner solve at each ρ trial and supplies the dense per-row Hessian block for the outer trace terms.
Formula → design integration — build_formula_design_for_multinomial parses the Wilkinson formula and assembles X and the per-term S blocks; the fit_multinomial_formula_pyfunc FFI shim wires the Python gamfit.fit(..., family='multinomial') entry straight to this path.

§Convergence

The damped-Newton-with-backtracking scaffold lives once in the shared crate::penalized_vector_glm engine: at each iteration the assembled penalized Hessian H + I_{K-1} ⊗ (λ_a S) is factored via faer’s symmetric-PD-with-fallback path, the full Newton step δ = −H^{-1} ∇F is computed, and accepted with step halving if the objective fails to decrease (up to a small backtracking budget). Convergence requires both a relative coefficient step ‖δ‖ / (1 + ‖β‖) ≤ tol and a fresh first-order score certificate at the accepted final iterate; failure produces checkpoint evidence, never coefficients/covariance behind a false flag. This module is the softmax adapter over that engine: it supplies the dense (K-1)×(K-1) Fisher block, the residual, and the log-likelihood through MultinomialLogitLikelihood, and owns the class-count / simplex preconditions. The independent-binomial sibling crate::binomial_multi is the same engine with a row-diagonal Fisher block instead.

Structs§

MultinomialFitInputs: Extra evidence used only for a NON-CONVERGED capped unbiased probe.
MultinomialFitOutputs: Outputs of fit_penalized_multinomial.
MultinomialFitRequest: Canonical typed inputs for the formula-driven multinomial fit (fit_penalized_multinomial_formula).
MultinomialModelEnvelope: Round-trip persistence envelope for a fitted multinomial model. The model_class discriminator lets a loader tell a multinomial payload apart from the scalar FittedModel JSON before deserialising the whole struct.
MultinomialPredictionIntervals
MultinomialSavedModel: Saved-model payload for a multinomial fit driven by a Wilkinson formula.
MultinomialSmoothSignificance: One row of the multinomial smooth-significance table (#1101): the Wood rank-truncated Wald test for one (active class, smooth term) pair.
MultinomialSmoothTermSpan: One smooth term’s coefficient span within a class block, plus its unpenalized nullspace dimension and a display label (#1101). The Wald smooth-significance test in summary() slices the joint covariance / influence at a·P + col_start .. a·P + col_end for active class a.

Constants§

MULTINOMIAL_MODEL_CLASS: On-disk model_class discriminator for a persisted multinomial model. Kept as a single constant so every producer / consumer of the envelope agrees on the tag without a scattered string literal.
MULTINOMIAL_MODEL_FORMAT_VERSION: Exact multinomial persistence schema. Version 2 requires the canonical per-component lambda labels and training-table provenance; successful deserialization therefore yields a complete current model without repair.

Functions§

fit_penalized_multinomial: Fit a penalized multinomial-logit GAM at fixed λ.
fit_penalized_multinomial_formula: Top-level formula-driven multinomial fit.
posterior_predict_multinomial_formula: Draw n_draws posterior-predictive replicate class-label assignments for a saved multinomial model on fresh data (#1101). Rebuilds the predict design exactly as predict_multinomial_formula, then samples each row’s class from Categorical(E[softmax(η) | data]) (see MultinomialSavedModel::sample_replicate_classes). Returns an (n_draws, N) matrix of class INDICES 0..K aligned to model.class_levels, deterministic in seed.
predict_multinomial_formula: Replay the saved termspec to build the predict-time design on a fresh dataset, then evaluate softmax probabilities. The predict dataset must carry the same feature columns the training data did, matched by name — it need not reproduce the training column order, and in particular need not carry the response column (prediction is for label-free new data).
predict_multinomial_formula_with_intervals: Build simplex-clamped normal moment intervals around the integrated logistic-normal posterior mean. Both center and spread come from the same deterministic posterior integral; no plug-in/delta quantity enters.
predict_multinomial_formula_with_se: Predict posterior-mean class probabilities and integrated marginal standard deviations for a saved multinomial model on fresh data.