Struct NeuralNetwork 

pub struct NeuralNetwork<const IN: usize, const OUT: usize> { /* private fields */ }

Neural Network

This is the main struct of the library: a chain of fully connected layers, each with its own weights, biases and ActivationFunction. You can use this struct and its methods to create, manipulate and even implement your own ways to train a neural network.

The number of input neurons (IN) and output neurons (OUT) are part of the type, so feeding a wrongly sized input is a compile error rather than a runtime panic. The hidden layers stay dynamic and are given at construction time.

Layers

Layer 0 is the input layer, which only passes the inputs on, so it has no weights, biases or activation. Layers 1..num_layers() each hold one row of weights per neuron, with one weight per neuron of the previous layer, and one bias per neuron.

Example

use only_brain::NeuralNetwork;

// A 2 -> 2 -> 1 network.
let mut nn = NeuralNetwork::<2, 1>::new(&[2]);

// One row per neuron, one weight per neuron of the previous layer.
nn.set_layer_weights(1, &[[0.1, 0.2],
                          [0.3, 0.4]]);
nn.set_layer_biases(1, &[0.1, 0.2]);

nn.set_layer_weights(2, &[[0.9, 0.8]]);
nn.set_layer_biases(2, &[0.1]);

let output = nn.feed_forward(&[0.5, 0.2]);

println!("{:?}", output);

Flat parameter view

Search methods such as genetic algorithms usually work on a flat list of numbers rather than on layers. parameters, set_parameters and from_parameters convert between a network and such a list, in an order that is documented and stable across versions:

• layer by layer, from layer 1 to the output layer;
• within a layer, every weight first, row by row (all the weights of neuron 0, then of neuron 1, and so on, each row in the order of the previous layer’s neurons);
• then that layer’s biases, one per neuron.

Activation functions are part of the network’s shape, not of its parameters, so they are left untouched by set_parameters.

use only_brain::NeuralNetwork;

let mut nn = NeuralNetwork::<2, 1>::new(&[]);
nn.set_layer_weights(1, &[[0.5, -0.25]]);
nn.set_layer_biases(1, &[0.1]);

assert_eq!(nn.parameters(), vec![0.5, -0.25, 0.1]);

let rebuilt = NeuralNetwork::<2, 1>::from_parameters(&[], &nn.parameters());
assert_eq!(rebuilt, nn);

Serialization

Networks implement serde’s Serialize and Deserialize through a plain form, a list of layers each with its activation, weights (one row per neuron) and biases, so they can be embedded in your own types and formats. Deserializing checks that the layers fit together and match IN and OUT. To save a network to a file, see crate::dump_model and crate::load_model.

Implementations

impl<const IN: usize, const OUT: usize> NeuralNetwork<IN, OUT>

pub fn new(hidden: &[usize]) -> Self

Creates a new Neural Network with the given hidden layer sizes. The input and output widths come from the type parameters, so hidden lists only the layers between them and may be empty.

Weights are initialised uniformly at random in [-1, 1), biases at zero, and every layer uses ActivationFunction::Sigmoid. Use NeuralNetwork::new_with_rng to control the seed.

Panics

Panics if any hidden layer size is zero. IN or OUT being zero is a compile error.

Example

// 2 -> 2 -> 1
let nn = NeuralNetwork::<2, 1>::new(&[2]);

// 3 -> 1, no hidden layers
let direct = NeuralNetwork::<3, 1>::new(&[]);

pub fn new_with_rng<R: Rng>(hidden: &[usize], rng: &mut R) -> Self

Creates a new Neural Network using the given random number generator.

Seeding the generator makes initialisation reproducible, which is what you want in tests and when a training run needs to be repeatable.

Panics

Panics if any hidden layer size is zero.

pub fn from_parameters(hidden: &[usize], parameters: &[f64]) -> Self

Creates a network with the given hidden layer sizes from a flat list of parameters, in the order described in the flat parameter view. Every layer uses ActivationFunction::Sigmoid; change that afterwards with set_activation_function and friends.

Panics

Panics if any hidden layer size is zero, or if parameters does not hold exactly parameter_count_for(hidden) values.

Example

// A genome from a genetic algorithm, for a 2 -> 2 -> 1 network.
let genome = vec![0.5; NeuralNetwork::<2, 1>::parameter_count_for(&[2])];

let mut nn = NeuralNetwork::<2, 1>::from_parameters(&[2], &genome);
nn.set_output_activation(ActivationFunction::Tanh);

let [steering] = nn.feed_forward(&[0.3, -0.8]);
assert!((-1.0..=1.0).contains(&steering));

pub fn feed_forward(&self, inputs: &[f64; IN]) -> [f64; OUT]

Feeds the given inputs to the neural network and returns the output.

The input and output widths are checked at compile time.

Example

let mut nn = NeuralNetwork::<1, 1>::new(&[]);

nn.set_layer_weights(1, &[[0.5]]);
nn.set_layer_biases(1, &[0.5]);

let output = nn.feed_forward(&[0.5]);
assert!((output[0] - 0.679178699175393).abs() < 1e-12);

pub fn feed_forward_batch(&self, inputs: &[[f64; IN]]) -> Vec<[f64; OUT]>

Feeds every input to the network and returns one output per input, in order.

This computes the same values as calling feed_forward once per input, but runs each layer as a single matrix product over all the inputs at once, which is faster when there are many of them: scoring a dataset, for example. It does not help when every input goes to a different network.

Example

let nn = NeuralNetwork::<2, 1>::new(&[3]);

let inputs = [[0.0, 0.0], [0.0, 1.0], [1.0, 0.0], [1.0, 1.0]];
let outputs = nn.feed_forward_batch(&inputs);

assert_eq!(outputs.len(), 4);
assert_eq!(outputs[2], nn.feed_forward(&inputs[2]));

pub fn parameter_count(&self) -> usize

Returns the number of weights and biases in the network, which is the length of parameters.

pub fn parameter_count_for(hidden: &[usize]) -> usize

Returns the number of weights and biases a network with these hidden layer sizes has, without building one. This is the genome length to give a genetic algorithm.

Panics

Panics if any hidden layer size is zero.

Example

// 10 -> 8 -> 3: (10 + 1) * 8 + (8 + 1) * 3
assert_eq!(NeuralNetwork::<10, 3>::parameter_count_for(&[8]), 115);

pub fn parameters(&self) -> Vec<f64>

Returns every weight and bias as one flat list, in the order described in the flat parameter view.

pub fn set_parameters(&mut self, parameters: &[f64])

Replaces every weight and bias from one flat list, in the order described in the flat parameter view. Activation functions are kept.

Panics

Panics if parameters does not hold exactly parameter_count() values.

pub fn set_layer_weights<R: AsRef<[f64]>>( &mut self, layer: usize, weights: &[R], )

Sets every weight of the given layer, as one row per neuron of that layer holding one weight per neuron of the previous layer.

Layer 0 is the input layer, which has no weights, so layer starts at 1. Nested arrays and Vec<Vec<f64>> are both accepted.

Panics

Panics if layer is 0 or past the output layer, or if weights does not have exactly layer_size(layer) rows of layer_size(layer - 1) weights.

Example

let mut nn = NeuralNetwork::<3, 1>::new(&[2]);

nn.set_layer_weights(1, &[[0.1, 0.2, 0.3],
                          [0.4, 0.5, 0.6]]);

// Weights computed at runtime work the same way.
let output_weights = vec![vec![0.7, 0.8]];
nn.set_layer_weights(2, &output_weights);

assert_eq!(nn.layer_weights(2), output_weights);

pub fn layer_weights(&self, layer: usize) -> Vec<Vec<f64>>

Returns the weights of the given layer, as one row per neuron of that layer holding one weight per neuron of the previous layer.

Panics

Panics if layer is 0 or past the output layer.

pub fn set_layer_biases(&mut self, layer: usize, biases: &[f64])

Sets the biases of the given layer, one per neuron.

Panics

Panics if layer is 0 or past the output layer, or if biases does not have exactly layer_size(layer) elements.

pub fn layer_biases(&self, layer: usize) -> &[f64]

Returns the biases of the given layer, one per neuron.

Panics

Panics if layer is 0 or past the output layer.

pub fn set_weight( &mut self, layer: usize, neuron: usize, input: usize, weight: f64, )

Sets the weight of a specific neuron connection. The layer index must be greater than 0 since the input layer does not have weights.

pub fn get_weight(&self, layer: usize, neuron: usize, input: usize) -> f64

Gets the weight of a specific neuron connection. The layer index must be greater than 0 since the input layer does not have weights.

pub fn set_bias(&mut self, layer: usize, neuron: usize, bias: f64)

Sets the bias of a specific neuron. The layer index must be greater than 0 since the input layer does not have biases.

pub fn get_bias(&self, layer: usize, neuron: usize) -> f64

Gets the bias of a specific neuron. The layer index must be greater than 0 since the input layer does not have biases.

pub fn num_layers(&self) -> usize

Returns the number of layers of the neural network, counting the input layer.

pub fn layer_size(&self, layer: usize) -> usize

Returns the number of neurons of the given layer. Layer 0 is the input layer.

pub fn hidden_layer_sizes(&self) -> Vec<usize>

Returns the sizes of the hidden layers, the same list given to new or from_parameters.

let nn = NeuralNetwork::<4, 2>::new(&[8, 6]);

let copy = NeuralNetwork::<4, 2>::from_parameters(&nn.hidden_layer_sizes(), &nn.parameters());
assert_eq!(copy.hidden_layer_sizes(), vec![8, 6]);

pub fn layer_activation(&self, layer: usize) -> ActivationFunction

Returns the activation function of the given layer.

Panics

Panics if layer is 0, since the input layer has no activation, or past the output layer.

pub fn set_layer_activation( &mut self, layer: usize, activation_function: ActivationFunction, )

Sets the activation function of the given layer.

Panics

Panics if layer is 0, since the input layer has no activation, or past the output layer.

pub fn set_activation_function( &mut self, activation_function: ActivationFunction, )

Sets the activation function of every layer of the network.

Combine it with set_output_activation to give the hidden layers and the output layer different functions.

Example

let mut nn = NeuralNetwork::<2, 1>::new(&[3]);
nn.set_activation_function(ActivationFunction::ReLU);
nn.set_output_activation(ActivationFunction::Tanh);

assert_eq!(nn.layer_activation(1), ActivationFunction::ReLU);
assert_eq!(nn.layer_activation(2), ActivationFunction::Tanh);

pub fn set_output_activation(&mut self, activation_function: ActivationFunction)

Sets the activation function of the output layer only.

pub fn output_activation(&self) -> ActivationFunction

Returns the activation function of the output layer.

pub fn print(&self)

Trait Implementations

impl<const IN: usize, const OUT: usize> Clone for NeuralNetwork<IN, OUT>

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<const IN: usize, const OUT: usize> Debug for NeuralNetwork<IN, OUT>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de, const IN: usize, const OUT: usize> Deserialize<'de> for NeuralNetwork<IN, OUT>

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl<const IN: usize, const OUT: usize> Display for NeuralNetwork<IN, OUT>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<const IN: usize, const OUT: usize> PartialEq for NeuralNetwork<IN, OUT>

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl<const IN: usize, const OUT: usize> Serialize for NeuralNetwork<IN, OUT>

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl<const IN: usize, const OUT: usize> StructuralPartialEq for NeuralNetwork<IN, OUT>