Function gradient

pub fn gradient<V>(f: &impl Fn(&V) -> f64, x0: &V, h: Option<f64>) -> V
where
    V: Vector<f64>,

Gradient of a multivariate, scalar-valued function using the forward difference approximation.

Arguments

• f - Multivariate, scalar-valued function, $f:\mathbb{R}^{n}\to\mathbb{R}$.
• x0 - Evaluation point, $\mathbf{x}_{0}\in\mathbb{R}^{n}$.
• h - Relative step size, $h\in\mathbb{R}$. Defaults to SQRT_EPS.

Returns

Gradient of $f$ with respect to $\mathbf{x}$, evaluated at $\mathbf{x}=\mathbf{x}_{0}$.

$$\nabla f(\mathbf{x}_{0})\in\mathbb{R}^{n}$$

Note

This function performs $n+1$ evaluations of $f(\mathbf{x})$.

Example

Approximate the gradient of

$$f(\mathbf{x})=x_{0}^{5}+\sin^{3}{x_{1}}$$

at $\mathbf{x}=(5,8)^{T}$, and compare the result to the true result of

$$ \nabla f\left((5,8)^{T}\right)= \begin{bmatrix} 3125 \\ 3\sin^{2}{(8)}\cos{(8)} \end{bmatrix} $$

Using standard vectors

use numtest::*;

use numdiff::forward_difference::gradient;

// Define the function, f(x).
let f = |x: &Vec<f64>| x[0].powi(5) + x[1].sin().powi(3);

// Define the evaluation point.
let x0 = vec![5.0, 8.0];

// Approximate the gradient of f(x) at the evaluation point.
let grad: Vec<f64> = gradient(&f, &x0, None);

// True gradient of f(x) at the evaluation point.
let grad_true: Vec<f64> = vec![3125.0, 3.0 * 8.0_f64.sin().powi(2) * 8.0_f64.cos()];

// Check the accuracy of the gradient approximation.
assert_arrays_equal_to_decimal!(grad, grad_true, 4);

Using other vector types

We can also use other types of vectors, such as nalgebra::SVector, nalgebra::DVector, ndarray::Array1, faer::Mat, or any other type of vector that implements the linalg_traits::Vector trait.

use faer::Mat;
use linalg_traits::Vector;  // to provide from_slice method for faer::Mat
use nalgebra::{dvector, DVector, SVector};
use ndarray::{array, Array1};
use numtest::*;

use numdiff::forward_difference::gradient;

let grad_true: Vec<f64> = vec![3125.0, 3.0 * 8.0_f64.sin().powi(2) * 8.0_f64.cos()];

// nalgebra::DVector
let f_dvector = |x: &DVector<f64>| x[0].powi(5) + x[1].sin().powi(3);
let x0_dvector: DVector<f64> = dvector![5.0, 8.0];
let grad_dvector: DVector<f64> = gradient(&f_dvector, &x0_dvector, None);
assert_arrays_equal_to_decimal!(grad_dvector, grad_true, 4);

// nalgebra::SVector
let f_svector = |x: &SVector<f64,2>| x[0].powi(5) + x[1].sin().powi(3);
let x0_svector: SVector<f64, 2> = SVector::from_row_slice(&[5.0, 8.0]);
let grad_svector: SVector<f64, 2> = gradient(&f_svector, &x0_svector, None);
assert_arrays_equal_to_decimal!(grad_svector, grad_true, 4);

// ndarray::Array1
let f_array1 = |x: &Array1<f64>| x[0].powi(5) + x[1].sin().powi(3);
let x0_array1: Array1<f64> = array![5.0, 8.0];
let grad_array1: Array1<f64> = gradient(&f_array1, &x0_array1, None);
assert_arrays_equal_to_decimal!(grad_array1, grad_true, 4);

// faer::Mat
let f_mat = |x: &Mat<f64>| x[(0, 0)].powi(5) + x[(1, 0)].sin().powi(3);
let x0_mat: Mat<f64> = Mat::from_slice(&[5.0, 8.0]);
let grad_mat: Mat<f64> = gradient(&f_mat, &x0_mat, None);
assert_arrays_equal_to_decimal!(grad_mat.as_slice(), grad_true, 4);

Modifying the relative step size

We can also modify the relative step size. Choosing a coarser relative step size, we get a worse approximation.

use numtest::*;

use numdiff::forward_difference::gradient;

let f = |x: &Vec<f64>| x[0].powi(5) + x[1].sin().powi(3);
let x0 = vec![5.0, 8.0];

let grad: Vec<f64> = gradient(&f, &x0, Some(0.001));
let grad_true: Vec<f64> = vec![3125.0, 3.0 * 8.0_f64.sin().powi(2) * 8.0_f64.cos()];

assert_arrays_equal_to_decimal!(grad, grad_true, -1);