hal-ml 0.2.0

HAL: a machine learning library that is able to run on Nvidia, OpenCL or CPU BLAS based compute backends. It currently provides stackable classical neural networks, RNN's and soon to be LSTM's. A differentiation of this package is that we are looking to implement RTRL (instead of just BPTT) for the recurrent layers in order to provide a solid framework for online learning. We will also (in the future) be implementing various layers such as unitary RNN's, NTM's and Adaptive Computation time based LSTM's. HAL also comes with the ability to plot and do many basic math operations on arrays.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53

# HAL : Hyper Adaptive Learning
Rust based Cross-GPU Machine Learning. [![Build Status](https://travis-ci.org/jramapuram/hal.svg?branch=feature%2Flstm)](https://travis-ci.org/jramapuram/hal)

# Why Rust?
This project is for those that miss strongly typed compiled languages.
Rust was chosen specifically because of [this](http://www.oreilly.com/programming/free/files/why-rust.pdf)
Furthermore, we can offer fine grained control of your operations.
This means being able to grab dimensions of tensors at any stage, none of that unknown shape nonsense.
We can also micro-control steps. An example of this working with each individual forward timesteps on say an LSTM. Usually these are controlled by inner loops [Theano/Tensorflow, etc].

# Features
  - Multi GPU [model based] support
  - **OpenCL + CUDA + Parallel CPU support**
  - **LSTM's with internal RTRL [Work in Progress]**
  - **RNN's [Work in Progress]**
  - Perceptrons, AutoEncoders, ConvNets**[TODO]**
  - Optimizers:      [SGD, Adam, AdaGrad**[TODO]**]
  - Activations:     [Linear, Sigmoid, Tanh, ReLU, LReLU, Softmax]
  - Initializations: [Lecun Uniform, Glorot Normal, Glorot Uniform, Normal, Uniform]
  - Data Gatherers:  [SinSource, MNIST**[In Progress]**, CIFAR10**[TODO]**]
  - Loss Functions:  [MSE, L2, Cross-Entropy]
  - OpenGL based plotting and image loading, see [here](https://www.accelereyes.com/arrayfire/c/page_gfx.htm) for more info
  - Multi GPU [horizontal] support **[TODO]**

# Requirements
  - Rust 1.8 +
  - arrayfire libs preinstalled (or you can compile manually by following instructions below)
```bash
cargo run --example autoencoder
```

# Installation
See [here](docs/installation.md)

# Testing
HAL utilizes RUST's test framework to extensively test all of our modules.
We employ gradient checking on individual functions as well as layers.
Testing needs to run on one thread due to our device probing methods.
Furthermore, graphics **needs** to be disabled for testing [glfw issue].
```bash
AF_DISABLE_GRAPHICS=1 RUST_TEST_THREADS=1 cargo test
```

If you would like to see the results of the test (as well as benchmarks) run:
```bash
AF_DISABLE_GRAPHICS=1 RUST_TEST_THREADS=1 cargo test -- --nocapture
```

# Credits
  - Thanks to the [arrayfire](http://arrayfire.com/) team for working with me to get the [rust bindings](https://github.com/arrayfire/arrayfire-rust) up.
  - [Keras](https://github.com/fchollet/keras) for inspiration as a lot of functions are similar to their implementation (minus the theano nonsense).
  - Dr. Felix Gers for his insight into internal RTRL
  - Dr. Sepp Hochreiter for advise on LSTM's