1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
//! `scarecrow` is a basic and simple implementation of an artificial
//! neural network.
//!
//! ## Example
//!
//! This demonstrates the capability of the neural network to learn a
//! non-linear function, namely [XOR](https://en.wikipedia.org/wiki/Exclusive_or).
//! It trains on a truth-table using
//! [gradient descent](https://en.wikipedia.org/wiki/Stochastic_gradient_descent).
//!
//! First we define inputs `X` and targets `T`:
//!
//! ```
//! // Two binary input values, 4 possible combinations
//! let inputs = vec![0.0, 0.0,
//! 0.0, 1.0,
//! 1.0, 0.0,
//! 1.0, 1.0];
//! // Four binary output targets, one for each possible input value
//! let targets = vec![0.0,
//! 1.0,
//! 1.0,
//! 0.0];
//! ```
//!
//! Then, we construct a neural network by adding a number of layers
//! to a list:
//!
//! ```rust,ignore
//! let mut layers: LinkedList<Box<WeightedLayer>> = LinkedList::new();
//! // We start by a hidden "dense" layer of 6 neurons which should
//! // accept 2 input values.
//! layers.push_back(Box::new(DenseLayer::random(2, 6)));
//! // We attach hyperbolic activation functions to the dense layer
//! layers.push_back(Box::new(HyperbolicLayer { size: 6 }));
//! // We follow this with a final "dense" layer with a single neuron,
//! // expecting 6 inputs from the preceeding layer.
//! layers.push_back(Box::new(DenseLayer::random(6, 1)));
//! // This will be output neuron so we attach a sigmoid activation function
//! // to get an output between 0 and 1.
//! layers.push_back(Box::new(SigmoidLayer { size: 1 }));
//! ```
//!
//! Since this is before training, we should expect a completely
//! random output from the network. This can be seen by feeding the
//! inputs through the network:
//!
//! ```rust,ignore
//! for (x, t) in inputs.chunks(2).zip(targets.chunks(1)) {
//! let mut o = x.to_vec();
//! for l in layers.iter() {
//! o = l.output(&o);
//! }
//! println!("X: {:?}, Y: {:?}, T: {:?}", x, o, t);
//! }
//! ```
//!
//! Example of network output `Y`:
//!
//! ```text
//! X: [0, 0], Y: [0.4244223], T: [0]
//! X: [0, 1], Y: [0.049231697], T: [1]
//! X: [1, 0], Y: [0.12347225], T: [1]
//! X: [1, 1], Y: [0.02869209], T: [0]
//! ```
//!
//! To train the network, first create a suitable trainer and then
//! call its train method:
//!
//! ```rust,ignore
//! // A trainer which uses stochastic gradient descent. Run for
//! // 1000 iterations with a learning rate of 0.1.
//! let trainer = SGDTrainer::new(1000, 0.1);
//! // Train the network on the given inputs and targets
//! trainer.train(&mut layers, &inputs, &targets);
//! ```
//!
//! Now calculate the output for the trained network:
//!
//! ```rust,ignore
//! for (x, t) in inputs.chunks(2).zip(targets.chunks(1)) {
//! let mut o = x.to_vec();
//! for l in layers.iter() {
//! o = l.output(&o);
//! }
//! println!("X: {:?}, Y: {:?}, T: {:?}", x, o, t);
//! }
//! ```
//!
//! Final result, note that network output `Y` is quite close to
//! targets `T`:
//!
//! ```text
//! X: [0, 0], Y: [0.03515992], T: [0]
//! X: [0, 1], Y: [0.96479124], T: [1]
//! X: [1, 0], Y: [0.96392107], T: [1]
//! X: [1, 1], Y: [0.03710678], T: [0]
//! ```
extern crate rand;