smyl 0.1.3

Artificial Neuronal Network in Rust
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130

# Smyl - A Machine Learning Library in Rust

Smyl is a machine learning library written in Rust, providing a set of tools and abstractions for building and training
neural networks.

## Features

- **Matrix Operations**: Smyl provides a `Matrix` struct for efficient matrix operations, including addition,
  subtraction, multiplication, and more.
- **Activation Functions**: Smyl includes common activation functions such as ReLU and Sigmoid.
- **Layers**: Smyl defines two main layer types: `SignalLayer` and `SynapseLayer`, which can be used to build neural
  network architectures.
- **Macros (optional)**: With the `macros` feature enabled, Smyl provides a set of macros to simplify the creation of
  neural networks.
- **Idx3 Support (optional)**: With the `idx3` feature enabled, Smyl can read and process data in the IDX3 format,
  commonly used for storing images.

## Getting Started

To use Smyl, add the following to your `Cargo.toml` file:

```toml
[dependencies]
smyl = "0.1.1"
```

Then, in your Rust code, you can start with this hello world example

```rust
extern crate rand;
extern crate smyl;

use rand::{thread_rng, Rng};
use smyl::layer::LayerGradient;
use smyl::prelude::*;

fn main() {
    let mut rng = thread_rng();

    let input: SignalLayer<f64, 4> = Matrix([[1.0, 0.0, 1.0, 0.5]]);
    let expected: SignalLayer<f64, 4> = Matrix([[1.0, 0.0, 0.5, 0.75]]);
    let learning_rate: f64 = 0.01;

    let mut synapses1: SynapseLayer<f64, 4, 2, ReLU> = rng.gen();
    let mut synapses2: SynapseLayer<f64, 2, 2, ReLU> = rng.gen();
    let mut synapses3: SynapseLayer<f64, 2, 4, ReLU> = rng.gen();

    let total_epochs = 1000_000;
    let chunk_epochs = 1;

    for _ in 0..total_epochs / chunk_epochs {
        let mut gradient1 = LayerGradient::default();
        let mut gradient2 = LayerGradient::default();
        let mut gradient3 = LayerGradient::default();

        for _ in 0..chunk_epochs {
            let output1 = synapses1.forward(input);
            let output2 = synapses2.forward(output1);
            let output3 = synapses3.forward(output2);

            let output_gradient3 = output3 - expected;
            let output_gradient2 =
                synapses3.backward(output2, output3, output_gradient3, &mut gradient3);
            let output_gradient1 =
                synapses2.backward(output1, output2, output_gradient2, &mut gradient2);
            synapses1.backward(input, output1, output_gradient1, &mut gradient1);

            dbg!(output3);
        }

        synapses3.apply_gradient(gradient3, learning_rate);
        synapses2.apply_gradient(gradient2, learning_rate);
        synapses1.apply_gradient(gradient1, learning_rate);
    }
}
```

But you might prefer using the macro:

```rust
use smyl::prelude::*;

fn main() {
    let mut ann = ann!(4, 6, 4);

    let input = Matrix([[1.0, 0.0, 1.0, 0.5]]);
    let expected = Matrix([[1.0, 0.0, 0.0, 0.0]]);
    let batch = 100;
    let learning_rate = 0.01;

    for i in 0..10000 {
        let output = ann.forward(input);
        ann.backward(expected);

        if i % batch == 0 {
            ann.apply_chunk_gradient(learning_rate * batch as f64);

            println!("{output:?}");
        }
    }
}
```

## Examples

You can find more example usage of Smyl in the `examples` directory of the repository:

- [**Hello World**](examples/hello.rs): A simple example demonstrating the basic usage of the library.
- [**Macros**](examples/macro.rs): An example showcasing the use of macros for creating neural networks.
- [**MNIST**](examples/mnist/main.rs): A more complex example that demonstrates training a neural network on the MNIST
  dataset.

# Documentation

For more detailed documentations, please refer to the [docs.rs](https://docs.rs/smyl/latest/smyl/) page.

## Contributing

Contributions are welcome! Please feel free to submit a pull request or open an issue for any suggestions or
improvements.

## License

This project is licensed under the GNU GPLv3 License. See the [LICENSE](LICENSE) file for details.

## Acknowledgments

- Thanks to the Rust community for their support and contributions.
- Special thanks to the [3Blue1Brown](https://www.3blue1brown.com/) channel for vulgarizing
  mathematics involved in machine learning