Struct egobox_gp::SparseGaussianProcess

pub struct SparseGaussianProcess<F: Float, Corr: CorrelationModel<F>> { /* private fields */ }

Sparse gaussian process considers a set of M inducing points either to approximate the posterior Gaussian distribution with a low-rank representation (FITC - Fully Independent Training Conditional method), or to approximate the posterior distribution directly (VFE - Variational Free Energy method).

These methods enable accurate modeling with large training datasets of N points while preserving computational efficiency. With M < N, we get O(NM^2) complexity instead of O(N^3) in time processing and O(NM) instead of O(N^2) in memory space.

See Reference section for more information.

Implementation

SparseGaussianProcess inducing points definition can be either random or provided by the user through the Inducings specification. The used sparse method is specified with the SparseMethod. Noise variance can be either specified as a known constant or estimated (see [ParamEstimation]). Unlike GaussianProcess implementation SparseGaussianProcess does not allow choosing a trend which is supposed to be zero. The correlation kernel might be selected amongst available kernels. When targetting a squared exponential kernel, one can use the SparseKriging shortcut.

Features

serializable

The serializable feature enables the serialization of GP models using the serde crate.

Example

use ndarray::{Array, Array2, Axis};
use ndarray_rand::rand;
use ndarray_rand::rand::SeedableRng;
use ndarray_rand::RandomExt;
use ndarray_rand::rand_distr::{Normal, Uniform};
use linfa::prelude::{Dataset, DatasetBase, Fit, Float, PredictInplace};

use egobox_gp::{SparseKriging, Inducings};

const PI: f64 = std::f64::consts::PI;

// Let us define a hidden target function for our sparse GP example
fn f_obj(x: &Array2<f64>) -> Array2<f64> {
  x.mapv(|v| (3. * PI * v).sin() + 0.3 * (9. * PI * v).cos() + 0.5 * (7. * PI * v).sin())
}

// Then we can define a utility function to generate some noisy data
// nt points with a gaussian noise with a variance eta2.
fn make_test_data(
    nt: usize,
    eta2: f64,
) -> (Array2<f64>, Array2<f64>) {
    let normal = Normal::new(0., eta2.sqrt()).unwrap();
    let mut rng = rand::thread_rng();
    let gaussian_noise = Array::<f64, _>::random_using((nt, 1), normal, &mut rng);
    let xt = 2. * Array::<f64, _>::random_using((nt, 1), Uniform::new(0., 1.), &mut rng) - 1.;
    let yt = f_obj(&xt) + gaussian_noise;
    (xt, yt)
}

// Generate training data
let nt = 200;
// Variance of the gaussian noise on our training data
let eta2: f64 = 0.01;
let (xt, yt) = make_test_data(nt, eta2);

// Train our sparse gaussian process with n inducing points taken in the dataset
let n_inducings = 30;
let sgp = SparseKriging::params(Inducings::Randomized(n_inducings))
    .fit(&Dataset::new(xt, yt))
    .expect("SGP fitted");

println!("sgp theta={:?}", sgp.theta());
println!("sgp variance={:?}", sgp.variance());
println!("noise variance={:?}", sgp.noise_variance());

// Predict with our trained SGP
let xplot = Array::linspace(-1., 1., 100).insert_axis(Axis(1));
let sgp_vals = sgp.predict(&xplot).unwrap();
let sgp_vars = sgp.predict_var(&xplot).unwrap();

Reference

Matthias Bauer, Mark van der Wilk, and Carl Edward Rasmussen. Understanding Probabilistic Sparse Gaussian Process Approximations. In: Advances in Neural Information Processing Systems. Ed. by D. Lee et al. Vol. 29. Curran Associates, Inc., 2016

Implementations

impl<F: Float, Corr: CorrelationModel<F>> SparseGaussianProcess<F, Corr>

pub fn params<NewCorr: CorrelationModel<F>>( corr: NewCorr, inducings: Inducings<F> ) -> SgpParams<F, NewCorr>

Gp parameters contructor

pub fn predict( &self, x: &ArrayBase<impl Data<Elem = F>, Ix2> ) -> Result<Array2<F>>

Predict output values at n given x points of nx components specified as a (n, nx) matrix. Returns n scalar output values as (n, 1) column vector.

pub fn predict_var( &self, x: &ArrayBase<impl Data<Elem = F>, Ix2> ) -> Result<Array2<F>>

Predict variance values at n given x points of nx components specified as a (n, nx) matrix. Returns n variance values as (n, 1) column vector.

pub fn kpls_dim(&self) -> Option<usize>

Retrieve number of PLS components 1 <= n <= x dimension

pub fn input_dim(&self) -> usize

Retrieve input dimension before kpls dimension reduction if any

pub fn output_dim(&self) -> usize

Retrieve output dimension

pub fn theta(&self) -> &Array1<F>

Optimal theta

pub fn variance(&self) -> F

Estimated variance

pub fn noise_variance(&self) -> F

Estimated noise variance

pub fn likelihood(&self) -> F

Retrieve reduced likelihood value

pub fn inducings(&self) -> &Array2<F>

Inducing points

pub fn predict_gradients( &self, x: &ArrayBase<impl Data<Elem = F>, Ix2> ) -> Array2<F>

pub fn predict_var_gradients( &self, x: &ArrayBase<impl Data<Elem = F>, Ix2> ) -> Array2<F>

pub fn sample_chol( &self, x: &ArrayBase<impl Data<Elem = F>, Ix2>, n_traj: usize ) -> Array2<F>

Sample the gaussian process for n_traj trajectories using cholesky decomposition

pub fn sample_eig( &self, x: &ArrayBase<impl Data<Elem = F>, Ix2>, n_traj: usize ) -> Array2<F>

Sample the gaussian process for n_traj trajectories using eigenvalues decomposition

pub fn sample( &self, x: &ArrayBase<impl Data<Elem = F>, Ix2>, n_traj: usize ) -> Array2<F>

Sample the gaussian process for n_traj trajectories using eigenvalues decomposition (alias of sample_eig)

Trait Implementations

impl<F: Float, Corr: CorrelationModel<F>> Clone for SparseGaussianProcess<F, Corr>

fn clone(&self) -> Self

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<F: Debug + Float, Corr: Debug + CorrelationModel<F>> Debug for SparseGaussianProcess<F, Corr>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<F: Float, Corr: CorrelationModel<F>> Display for SparseGaussianProcess<F, Corr>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<F, D, Corr> PredictInplace<ArrayBase<D, Dim<[usize; 2]>>, ArrayBase<OwnedRepr<F>, Dim<[usize; 2]>>> for SparseGaussianProcess<F, Corr>

where F: Float, D: Data<Elem = F>, Corr: CorrelationModel<F>,

fn predict_inplace(&self, x: &ArrayBase<D, Ix2>, y: &mut Array2<F>)

Predict something in place

fn default_target(&self, x: &ArrayBase<D, Ix2>) -> Array2<F>

Create targets that predict_inplace works with.