Crate egobox_moe 

This library implements Mixture of Experts method using GP models.

MoE method aims at increasing the accuracy of a function approximation by replacing a single global model by a weighted sum of local gp regression models (experts). It is based on a partition of the problem domain into several subdomains via clustering algorithms followed by a local expert training on each subdomain.

The recombination between the GP models can be either:

• hard: one GP model is being responsible to provide the predicted value at the given point. GP selection is done by taking the largest probability of the given point being part of the cluster corresponding to the expert GP. In hard mode, transition between models leads to discontinuity.
• smooth: all GPs models are taken and their predicted values at a given point are weighted regarding their responsability (probability of the given point being part of the cluster corresponding to the expert GP). In this case the MoE model is continuous. The smoothness is automatically adjusted using a factor, the heaviside factor, which can also be set manually.

Implementation

• Clusters are defined by clustering the training data with linfa-clustering gaussian mixture model.
• This library is a port of the SMT MoE method using egobox GP models as experts.
• It leverages on the egobox GP PLS reduction feature to handle high dimensional problems.
• MoE trained model can be save to disk and reloaded. See

Features

serializable

The serializable feature enables serialization based on serde crate.

persistent

The persistent feature enables save()/load() methods for a MoE model to/from a json file using the serde and serde_json crates.

Example

use ndarray::{Array2, Array1, Zip, Axis};
use egobox_moe::{GpMixture, Recombination, NbClusters};
use ndarray_rand::{RandomExt, rand::SeedableRng, rand_distr::Uniform};
use rand_xoshiro::Xoshiro256Plus;
use linfa::{traits::Fit, ParamGuard, Dataset};

// One-dimensional test function with 3 modes
fn f3modes(x: &Array1<f64>) -> Array1<f64> {
    let mut y = Array1::zeros(x.len());
    Zip::from(&mut y).and(x).for_each(|yi, &xi| {
        if xi < 0.4 {
            *yi = xi * xi;
        } else if (0.4..0.8).contains(&xi) {
            *yi = 3. * xi + 1.;
        } else {
            *yi = f64::sin(10. * xi);
        }
    });
    y
}

// Training data
let mut rng = Xoshiro256Plus::from_entropy();
let xt = Array1::random_using((50, ), Uniform::new(0., 1.), &mut rng);
let yt = f3modes(&xt);
let ds = Dataset::new(xt.insert_axis(Axis(1)), yt);

// Predictions
let observations = Array1::linspace(0., 1., 100).insert_axis(Axis(1));
let predictions = GpMixture::params()
                    .n_clusters(NbClusters::fixed(3))
                    .recombination(Recombination::Hard)
                    .fit(&ds)
                    .expect("MoE model training")
                    .predict(&observations)
                    .expect("MoE predictions");

Reference

Bettebghor, Dimitri, et al. Surrogate modeling approximation using a mixture of experts based on EM joint estimation Structural and multidisciplinary optimization 43.2 (2011): 243-259.

Structs

Clustering

A structure for clustering

CorrelationSpec

Flags to specify tested correlation models during experts selection (see correlation_spec()).

GaussianMixture

Gaussian mixture is a set of n weigthed multivariate normal distributions of dimension nx This structure is derived from linfa::GaussianMixtureModel clustering method to handle the resulting multivariate normals and related computations in one go. Moreover an heaviside factor is handled in case of smooth Recombination to control the smoothness between clusters and can be adjusted afterwards

GpConstantAbsoluteExponentialSurrogate

GP surrogate with Constant regression model and AbsoluteExponential correlation model.

GpConstantMatern32Surrogate

GP surrogate with Constant regression model and Matern32 correlation model.

GpConstantMatern52Surrogate

GP surrogate with Constant regression model and Matern52 correlation model.

GpConstantSquaredExponentialSurrogate

GP surrogate with Constant regression model and SquaredExponential correlation model.

GpLinearAbsoluteExponentialSurrogate

GP surrogate with Linear regression model and AbsoluteExponential correlation model.

GpLinearMatern32Surrogate

GP surrogate with Linear regression model and Matern32 correlation model.

GpLinearMatern52Surrogate

GP surrogate with Linear regression model and Matern52 correlation model.

GpLinearSquaredExponentialSurrogate

GP surrogate with Linear regression model and SquaredExponential correlation model.

GpMetricResult

Result of a GP quality assessment metric

GpMixture

Mixture of gaussian process experts Implementation note: the structure is not generic over ‘F: Float’ to be able to implement use serde easily as deserialization of generic impls is not supported yet See https://github.com/dtolnay/typetag/issues/1

GpMixtureParams

Mixture of experts parameters

GpMixtureValidParams

Mixture of experts checked parameters

GpQuadraticAbsoluteExponentialSurrogate

GP surrogate with Quadratic regression model and AbsoluteExponential correlation model.

GpQuadraticMatern32Surrogate

GP surrogate with Quadratic regression model and Matern32 correlation model.

GpQuadraticMatern52Surrogate

GP surrogate with Quadratic regression model and Matern52 correlation model.

GpQuadraticSquaredExponentialSurrogate

GP surrogate with Quadratic regression model and SquaredExponential correlation model.

IaeAlphaPlotData

Data structure to hold data for IAE alpha plot

MoeVariancePredictor

Adaptator to implement linfa::Predict for variance prediction

RegressionSpec

Flags to specify tested regression models during experts selection (see regression_spec()).

SgpAbsoluteExponentialSurrogate

SGP surrogate with AbsoluteExponential correlation model.

SgpMatern32Surrogate

SGP surrogate with Matern32 correlation model.

SgpMatern52Surrogate

SGP surrogate with Matern52 correlation model.

SgpSquaredExponentialSurrogate

SGP surrogate with SquaredExponential correlation model.

Enums

GpFileFormat

An enumeration of Gpx available file format

GpMetric

Enum of available metrics for Gaussian Process quality assessment

GpType

Type of Gaussian Process

Inducings

SGP inducing points specification

MoeError

An error when using MOE algorithm

NbClusters

Number of clusters specification

Recombination

Enumeration of recombination modes handled by the mixture

SparseMethod

SGP algorithm method specification

ThetaTuning

An enum to represent a n-dim hyper parameter tuning

Traits

Clustered

A trait to represent clustered structure

FullGpSurrogate

A trait for a GP surrogate.

GpMetrics

A trait for cross validation score

GpParameterized

A trait for a GP surrogate.

GpQualityAssurance

A trait for GP surrogate quality assessment

GpSurrogate

A trait for a base GP surrogate

GpSurrogateExt

A trait for a GP surrogate with derivatives predictions and sampling

GpSurrogateParams

A trait for Gp surrogate parameters to build surrogate.

MixtureGpSurrogate

A trait for Mixture of GP surrogates with derivatives using clustering

SgpSurrogate

A trait for a Sparse GP surrogate.

SgpSurrogateParams

A trait for sparse GP surrogate parameters to build surrogate.

Functions

find_best_number_of_clusters

Find the best number of cluster thanks to cross validation

Type Aliases

Result

A result type for Moe algorithm