Module problem 

Problem traits the user implements about their objective. Solvers bind on whichever subset they need (e.g. gradient descent requires CostFunction and Gradient; Nelder-Mead only needs CostFunction).

Traits

CostFunction

Scalar-valued objective f(x): Param → Output. The smallest problem trait — every solver binds at least on this.

Gradient

Analytic gradient ∇f(x): Param → Gradient. Required by first-order solvers (gradient descent, BFGS, …).

Hessian

Analytic Hessian H(x) = ∇²f(x): Param → Output for second-order solvers (Newton, trust-region-Newton). Like Jacobian, the associated Output matrix type lets solvers bound on the linear-algebra ops they need (LinearSolveSpd, SymmetricEigen, …) without baking in a backend.

Jacobian

Analytic Jacobian J(x) = ∂r/∂x: Param → Output for least-squares solvers (Gauss-Newton, LM, TRF). The associated Output matrix type is what lets solvers bound on the linear-algebra ops they need (MatVec, LinearSolveSpd, …) without baking in a specific backend or assuming density.

Residual

Vector-valued residual r(x): Param → Output for least-squares problems. Required by Gauss-Newton, Levenberg-Marquardt, and any solver that minimizes ½‖r(x)‖².