FFNN - a very simple feedforward neural network library

Not useful yet, still a WIP.

TODO

• Add documentation

• Backward propagation

• Batched backprop

• Adam optimizer

• Softmax activation

• Dropout

• Genetic algorithm utils?

• Multiple cost functions

  • Binary cross entropy
  • Categorical cross entropy
  • Mean squared error

• Maybe multiple optimization methods (not just gradient descent)

• Thread pool to increase performance

• Use GPU

Create some examples

• MNIST digits
• XOR function
• Sine wave

Done

• Forward propagation
• Add bias nodes
• Move initialization to the builder
• Multiple activation functions

References

• nalgebra docs
• The Math behind Neural Networks - Backpropagation
• Why Initialize a Neural Network with Random Weights?
• A Step by Step Backpropagation Example