Struct HigherOrderEngine 

pub struct HigherOrderEngine<T: Float + Debug + Default + Clone + Send + Sync + 'static + Sum + ScalarOperand> { /* private fields */ }

Higher-order differentiation engine

Implementations

impl<T: Float + Debug + Default + Clone + Send + Sync + 'static + Sum + ScalarOperand> HigherOrderEngine<T>

pub fn new(_maxorder: usize) -> Self

Create a new higher-order differentiation engine

pub fn with_config(config: HigherOrderConfig<T>) -> Self

Create a new engine with advanced configuration

pub fn set_mixed_mode(&mut self, enabled: bool)

Enable/disable mixed-mode computation

pub fn set_finite_diff_eps(&mut self, eps: T)

Set finite difference epsilon for numerical verification

pub fn hessian_forward_over_reverse( &mut self, function: impl Fn(&Array1<T>) -> T, point: &Array1<T>, config: &HessianConfig, ) -> Result<Array2<T>>

Compute Hessian matrix using forward-over-reverse mode.

When HessianConfig::sparse is set and the problem is large enough that sparsification pays for itself, entries below HessianConfig::sparsity_threshold are zeroed. Previously sparse/sparsity_threshold were accepted and ignored: the sparsity decision and the thresholding routine both existed but nothing called them.

The wall time is recorded on the engine’s profiler, so Self::get_derivative_stats’s performance_profile reports measured numbers rather than the zeros it returned when nothing was ever recorded.

pub fn hessian_reverse_over_forward( &mut self, function: impl Fn(&Array1<T>) -> T, point: &Array1<T>, _config: &HessianConfig, ) -> Result<Array2<T>>

Compute the Hessian column by column, by taking the gradient of each first partial derivative.

The name is historical — it describes the reverse-over-forward autodiff scheme this emulates, not the implementation, which is a nested central finite difference (see hvp for why autodiff cannot be threaded through this engine’s objective type). Distinct from Self::hessian_forward_over_reverse, which works row-wise from a directional derivative of the gradient.

Both nested steps use the order-2 step. With the old raw finite_diff_eps (1e-5) on both levels the roundoff floor was ε/h² ≈ 2e-6 in f64 and ≈ 1.2e3 in f32 — i.e. the f32 result was pure noise.

pub fn hessian_diagonal( &self, function: &impl Fn(&Array1<T>) -> T, point: &Array1<T>, ) -> Result<Array2<T>>

Compute diagonal Hessian elements only

pub fn sparse_hessian( &mut self, function: impl Fn(&Array1<T>) -> T, point: &Array1<T>, config: &HessianConfig, ) -> Result<SparseHessian<T>>

Compute sparse Hessian representation

pub fn third_order_derivatives( &mut self, function: impl Fn(&Array1<T>) -> T, point: &Array1<T>, ) -> Result<ThirdOrderTensor<T>>

Compute third-order derivatives (tensor)

pub fn mixed_partial( &mut self, function: impl Fn(&Array1<T>) -> T, point: &Array1<T>, variables: &[usize], orders: &[usize], method: MixedPartialMethod, ) -> Result<MixedPartials<T>>

Compute mixed partial derivatives

pub fn hessian_vector_product( &mut self, function: impl Fn(&Array1<T>) -> T, point: &Array1<T>, vector: &Array1<T>, ) -> Result<Array1<T>>

Compute Hessian-vector product efficiently

pub fn vector_hessian_vector_product( &mut self, function: impl Fn(&Array1<T>) -> T, point: &Array1<T>, vector: &Array1<T>, ) -> Result<T>

Compute vector-Hessian-vector product (quadratic form)

pub fn verify_derivatives( &self, function: impl Fn(&Array1<T>) -> T, point: &Array1<T>, computed_hessian: &Array2<T>, ) -> Result<DerivativeVerification<T>>

Verify higher-order derivatives using finite differences

pub fn get_derivative_stats(&self) -> HigherOrderStats

Get higher-order derivative statistics

pub fn hessian_vector_product_advanced( &mut self, function: impl Fn(&Array1<T>) -> T + Send + Sync, point: &Array1<T>, vector: &Array1<T>, mode: Option<HvpMode>, ) -> Result<Array1<T>>

Advanced Hessian-vector product with automatic mode selection

pub fn vector_hessian_vector_efficient( &mut self, function: impl Fn(&Array1<T>) -> T, point: &Array1<T>, vector: &Array1<T>, ) -> Result<T>

Compute vector-Hessian-vector efficiently

pub fn jacobian_efficient<F>( &mut self, function: F, point: &Array1<T>, output_dim: usize, ) -> Result<Array2<T>>

where F: Fn(&Array1<T>) -> Array1<T> + Send + Sync,

Compute Jacobian efficiently with automatic mode selection

pub fn kfac_hessian_approximation( &mut self, layers: &[LayerInfo<T>], activations: &[Array1<T>], gradients: &[Array1<T>], ) -> Result<Array2<T>>

Compute K-FAC approximation to the Hessian

pub fn natural_gradient( &mut self, log_likelihood: impl Fn(&Array1<T>) -> T, point: &Array1<T>, gradient: &Array1<T>, ) -> Result<Array1<T>>

Compute Natural Gradient using Fisher Information Matrix

pub fn truncated_newton_direction( &mut self, function: impl Fn(&Array1<T>) -> T + Send + Sync, point: &Array1<T>, gradient: &Array1<T>, max_cg_iterations: usize, cg_tolerance: T, ) -> Result<Array1<T>>

Compute truncated Newton direction