Peroxide

Pure Rust numeric library contains linear algebra, numerical analysis, statistics and machine learning tools with R, MATLAB, Python like macros.

Latest README version

Corresponding to 0.12.3.

Pre-requisite

Python module : matplotlib for plotting

Install

Add next line to your cargo.toml

peroxide = "0.12"

Module Structure

src
- lib.rs : mod and re-export
- macros : Macro files
  - julia_macro.rs : Julia like macro
  - matlab_macro.rs : MATLAB like macro
  - mod.rs
  - r_macro.rs : R like macro
- ml : For machine learning (Beta)
  - mod.rs
  - reg.rs : Regression tools
- numerical : To do numerical things
  - bdf.rs : Backward Differentiation Formula
  - gauss_legendre.rs : Gauss-Legendre 4th order
  - interp.rs : Interpolation
  - mod.rs
  - newton.rs : Newton's Method
  - ode.rs : Main ODE solver
  - optimize.rs : Non-linear regression
  - spline.rs : Natural Spline
  - utils.rs : Utils to do numerical things (e.g. jacobian)
- operation : To define general operations
  - extra_ops.rs : Missing operations & Real Trait
  - mut_ops.rs : Mutable operations
  - mod.rs
  - number.rs : Number type (include f64, Dual)
- special : Wrapper for special crate
  - mod.rs
  - function.rs : Special functions
- statistics : Statistical Tools
  - mod.rs
  - dist.rs : Probability distributions
  - ops.rs : Some probabilistic operations
  - rand.rs : Wrapper for rand crate
  - stat.rs : Statistical tools
- structure : Fundamental data structures
  - dataframe.rs : Not yet implemented
  - dual.rs : Dual number system for automatic differentiation
  - hyper_dual.rs : Hyper dual number system for automatic differentiation
  - matrix.rs : Matrix
  - multinomial.rs : For multinomial (Beta)
  - mod.rs
  - polynomial.rs : Polynomial
  - vector.rs : Extra tools for Vec<f64>
- util
  - mod.rs
  - api.rs : Matrix constructor for various language style
  - non_macro.rs : Primordial version of macros
  - pickle.rs : To handle pickle data structure
  - plot.rs : To draw plot (using pyo3)
  - print.rs : To print conveniently
  - useful.rs : Useful utils to implement library
  - writer.rs : More convenient write system

Documentation

Covered Contents

Linear Algebra
- Effective Matrix structure
- Transpose, Determinant, Diagonal
- LU Decomposition, Inverse matrix, Block partitioning
- Column, Row operations
Functional Programming
- More easy functional programming with Vec<f64>
- For matrix, there are three maps
  - fmap : map for all elements
  - col_map : map for column vectors
  - row_map : map for row vectors
Automatic Differentiation
- Dual number system - for 1st order AD
- Hyper dual number system - for 2nd order AD
- Exact jacobian
- Real trait to constrain for f64 and Dual
- Number structure to unify f64 and Dual
Numerical Analysis
- Lagrange interpolation
- Cubic spline
- Non-linear regression
  - Gradient Descent
  - Gauss Newton
  - Levenberg Marquardt
- Ordinary Differential Equation
  - Euler
  - Runge Kutta 4th order
  - Backward Euler
  - Gauss Legendre 4th order
Statistics
- More easy random with rand crate
- Probability Distributions
  - Bernoulli
  - Uniform
  - Normal
  - Gamma
  - Beta
- RNG algorithms
  - Acceptance Rejection
  - Marsaglia Polar
  - Ziggurat
Special functions
- Wrapper for special crate
Utils
- R-like macro & functions
- Matlab-like macro & functions
- Numpy-like macro & functions
- Julia-like macro & functions
Plotting
- With pyo3 & matplotlib

Example

Basic Runge-Kutta 4th order with inline-python

#![feature(proc_macro_hygiene)]
extern crate peroxide;
extern crate inline_python;
use peroxide::*;
use inline_python::python;

fn main() {
    // Initial condition
    let init_state = State::<f64>::new(0f64, c!(1), c!(0));

    let mut ode_solver = ExplicitODE::new(test_fn);

    ode_solver
        .set_method(ExMethod::RK4)
        .set_initial_condition(init_state)
        .set_step_size(0.01)
        .set_times(1000);

    let result = ode_solver.integrate();

    let x = result.col(0);
    let y = result.col(1);

    // Plot (Thanks to inline-python)
    python! {
        import pylab as plt
        plt.plot('x, 'y)
        plt.show()
    }
}

// dy/dx = (5x^2 - y) / e^(x+y)
fn test_fn(st: &mut State<f64>) {
    let x = st.param;
    let y = &st.value;
    let dy = &mut st.deriv;
    dy[0] = (5f64 * x.powi(2) - y[0]) / (x + y[0]).exp();
}

Basic Runge-Kutta 4th order with advanced plotting

extern crate peroxide;
use peroxide::*;

fn main() {
    let init_state = State::<f64>::new(0f64, c!(1), c!(0));

    let mut ode_solver = ExplicitODE::new(test_fn);

    ode_solver
        .set_method(ExMethod::RK4)
        .set_initial_condition(init_state)
        .set_step_size(0.01)
        .set_times(1000);

    let result = ode_solver.integrate();

    let x = result.col(0);
    let y = result.col(1);

    // Plot (using python matplotlib)
    let mut plt = Plot2D::new();
    plt.set_domain(x)
        .insert_image(y)
        .set_title("Test Figure")
        .set_fig_size((10, 6))
        .set_dpi(300)
        .set_legends(vec!["RK4"])
        .set_path("example_data/test_plot.png");

    plt.savefig();
}

fn test_fn(st: &mut State<f64>) {
    let x = st.param;
    let y = &st.value;
    let dy = &mut st.deriv;
    dy[0] = (5f64 * x.powi(2) - y[0]) / (x + y[0]).exp();
}

Basic Runge-Kutta 4th order with Stop condition

extern crate peroxide;
use peroxide::*;

fn main() {
    let init_state = State::<f64>::new(0f64, c!(1), c!(0));

    let mut ode_solver = ExplicitODE::new(test_fn);

    ode_solver
        .set_method(ExMethod::RK4)
        .set_initial_condition(init_state)
        .set_step_size(0.01)
        .set_stop_condition(stop)        // Add stop condition
        .set_times(1000);

    let result = ode_solver.integrate();

    let x = result.col(0);
    let y = result.col(1);

    let mut plt = Plot2D::new();
    plt.set_domain(x)
        .insert_image(y)
        .set_title("Test Figure")
        .set_fig_size((10, 6))
        .set_dpi(300)
        .set_legends(vec!["RK4"])
        .set_path("example_data/test_plot.png");

    plt.savefig();
}

fn test_fn(st: &mut State<f64>) {
    let x = st.param;
    let y = &st.value;
    let dy = &mut st.deriv;
    dy[0] = (5f64 * x.powi(2) - y[0]) / (x + y[0]).exp();
}

fn stop(st: &ExplicitODE) -> bool {
    let y = &st.get_state().value[0];
    (*y - 2.4).abs() < 0.01
}

Example image

Multi-Layer Perceptron (from scratch)

extern crate peroxide;
use peroxide::*;

// x : n x L
// xb: n x (L+1)
// v : (L+1) x M
// a : n x M
// ab: n x (M+1)
// w : (M+1) x n
// wb: M x N
// y : n x N
// t : n x N
// dh: n x M
// do: n x N

fn main() {
    let v = weights_init(3, 2);
    let w = weights_init(3, 1);

    let x = ml_matrix("0 0; 0 1; 1 0; 1 1");
    let t = ml_matrix("0;1;1;0");

    let y = train(v, w, x, t, 0.25, 5000);
    y.print();
}

fn weights_init(m: usize, n: usize) -> Matrix {
    rand(m, n) * 2f64 - 1f64
}

fn sigmoid(x: f64) -> f64 {
    1f64 / (1f64 + (-x).exp())
}

fn forward(weights: Matrix, input_bias: Matrix) -> Matrix {
    let s = input_bias * weights;
    s.fmap(|x| sigmoid(x))
}

fn add_bias(input: Matrix, bias: f64) -> Matrix {
    let b = matrix(vec![bias; input.row], input.row, 1, Col);
    cbind(b, input)
}

fn hide_bias(weight: Matrix) -> Matrix {
    weight.skip(1, Row)
}

fn train(
    weights1: Matrix,
    weights2: Matrix,
    input: Matrix,
    answer: Matrix,
    eta: f64,
    times: usize,
) -> Matrix {
    let x = input;
    let mut v = weights1;
    let mut w = weights2;
    let t = answer;
    let xb = add_bias(x.clone(), -1f64);

    for _i in 0..times {
        let a = forward(v.clone(), xb.clone());
        let ab = add_bias(a.clone(), -1f64);
        let y = forward(w.clone(), ab.clone());
        //        let err = (y.clone() - t.clone()).t() * (y.clone() - t.clone());
        let wb = hide_bias(w.clone());
        let delta_o = (y.clone() - t.clone()) * y.clone() * (1f64 - y.clone());
        let delta_h = (delta_o.clone() * wb.t()) * a.clone() * (1f64 - a.clone());

        w = w.clone() - eta * (ab.t() * delta_o);
        v = v.clone() - eta * (xb.t() * delta_h);
    }

    let a = forward(v, xb);
    let ab = add_bias(a, -1f64);
    let y = forward(w, ab);

    y
}

Levenberg-Marquardt Algorithm (refer to lm.pdf)

extern crate peroxide;
use peroxide::*;

fn main() {
    let noise = Normal(0., 0.5);
    let p_true: Vec<Number> = NumberVector::from_f64_vec(vec![20f64, 10f64, 1f64, 50f64]);
    let p_init = vec![5f64, 2f64, 0.2f64, 10f64];
    let domain = seq(0, 99, 1);
    let real = f(&domain, p_true.clone()).to_f64_vec();
    let y = zip_with(|x, y| x + y, &real, &noise.sample(100));
    let data = hstack!(domain.clone(), y.clone());

    let mut opt = Optimizer::new(data, f);
    let p = opt
        .set_init_param(p_init)
        .set_max_iter(100)
        .set_method(LevenbergMarquardt)
        .optimize();
    p.print();
    opt.get_error().print();
}

fn f(domain: &Vec<f64>, p: Vec<Number>) -> Vec<Number> {
    domain.clone().into_iter()
        .map(|t| Number::from_f64(t))
        .map(|t| p[0] * (-t / p[1]).exp() + p[2] * t * (-t / p[3]).exp())
        .collect()
}

Version Info

To see RELEASES.md

TODO

To see TODO.md

peroxide 0.13.0

Peroxide

Latest README version

Pre-requisite

Install

Module Structure

Documentation

Covered Contents

Example

Basic Runge-Kutta 4th order with inline-python

Basic Runge-Kutta 4th order with advanced plotting

Basic Runge-Kutta 4th order with Stop condition

Multi-Layer Perceptron (from scratch)

Levenberg-Marquardt Algorithm (refer to lm.pdf)

Version Info

TODO