hal-ml 0.2.0

HAL: a machine learning library that is able to run on Nvidia, OpenCL or CPU BLAS based compute backends. It currently provides stackable classical neural networks, RNN's and soon to be LSTM's. A differentiation of this package is that we are looking to implement RTRL (instead of just BPTT) for the recurrent layers in order to provide a solid framework for online learning. We will also (in the future) be implementing various layers such as unitary RNN's, NTM's and Adaptive Computation time based LSTM's. HAL also comes with the ability to plot and do many basic math operations on arrays.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82

pub use self::sin::SinSource;
mod sin;

pub use self::xor::XORSource;
mod xor;

use af::{Dim4, Array, DType};
use std::cell::{RefCell, Cell};

use utils;

#[derive(Clone)]
pub struct Data {
  pub input: RefCell<Box<Array>>,
  pub target: RefCell<Box<Array>>,
}

#[derive(PartialEq, Clone, Debug)]
pub struct DataParams {
  pub input_dims: Dim4,         // [batch_size, feature_x, feature_y, time]
  pub target_dims: Dim4,        // [batch_size, feature_x, feature_y, time]
  pub shuffle: bool,            // whether the data is shuffled
  pub normalize: bool,          // whether the data is normalized
  pub current_epoch: Cell<u64>, // for internal tracking of what the current epoch is
  pub dtype: DType,             // the type of data generated
  pub num_samples: u64,
  pub num_train: u64,
  pub num_test: u64,
  pub num_validation: Option<u64>,
}

/// A DataSource needs to provide these basic features
///
/// 1) It gives information regarding the source
/// 2) It provides a train iterator that returns a minibatch
/// 3) It provides a test iterator that returns a minibatch
/// 4) It (optionally)provides a validation iterator that returns a minibatch
pub trait DataSource {
  fn info(&self) -> DataParams;
  fn get_train_iter(&self, num_batch: u64) -> Data;
  fn get_test_iter(&self, num_batch: u64) -> Data;
  fn get_validation_iter(&self, num_batch: u64) -> Option<Data>;
}

/// Trait that describes a normalization operation
pub trait Normalize {
  fn normalize(&mut self, num_std: f32);
}

// TODO: Implement whitening via SVD
// pub trait Whiten {
//   fn whiten(&self);
// }

/// Trait that describes a shuffling operation
pub trait Shuffle {
  fn shuffle(&mut self);
}


// TODO: Implement this
impl Shuffle for Data {
  fn shuffle(&mut self) {
    //utils::shuffle_array(&mut[data.input.m, target], idims[0])
    //data.clone()
    println!("WARNING: shuffle not yet implemented");
  }
}

/// Implementation of the Normalize operation for Data
///
/// Currently only mean subtraction & std-deviation
/// division are supported. In the future we can
/// add whitening, etc.
impl Normalize for Data {
  fn normalize(&mut self, num_std: f32){
    let normalized_inputs = utils::normalize_array(&self.input.borrow(), num_std);
    let normalized_target = utils::normalize_array(&self.target.borrow(), num_std);
    self.input = RefCell::new(Box::new(normalized_inputs));
    self.target = RefCell::new(Box::new(normalized_target));
  }
}