# 3. Neural Networks
RustyML ships a small, Keras-shaped deep-learning framework, written in pure Rust. You stack layers into a [`Sequential`](./3.1._The_Sequential_Model.md) model, `compile` it with an optimizer and a loss, and call `fit`/`predict`. Every tensor that flows through the framework is a `Tensor`, which is just `ndarray::ArrayD<f32>`. It is single-precision, has a dynamic rank, and runs with no GPU and no autograd tape. Each layer implements forward and backward by hand, and hands its parameters to the optimizer through a flat view. This makes the whole framework deterministic and easy to debug. If you have used Keras, you will recognize the shape of the API. The differences are strict `f32` precision, explicit input dimensions, and `Result`-returning constructors, and this chapter explains them.
Read [Chapter 1](../Chapter-01/1.0._Getting_Started.md) before this chapter. Read [Working with ndarray](../Chapter-01/1.3._Working_with_ndarray.md) and [Installation and Feature Flags](../Chapter-01/1.2._Installation_and_Feature_Flags.md) too, since the `neural_network` feature gates this whole module. Read [Error Handling](../Chapter-01/1.6._Error_Handling.md) as well, since layer and loss constructors return `Result`. A model has this end-to-end shape:
```rust
use rustyml::neural_network::{
sequential::Sequential,
layers::{Activation, Dense},
optimizers::Adam,
losses::MeanSquaredError,
};
use ndarray::Array;
fn main() {
let x = Array::ones((8, 4)).into_dyn(); // 8 samples, 4 features
let y = Array::ones((8, 1)).into_dyn(); // 8 samples, 1 target
let mut model = Sequential::new();
model
.add(Dense::new(4, 16, Activation::ReLU).unwrap())
.add(Dense::new(16, 1, Activation::Linear).unwrap())
.compile(
Adam::new(0.001, 0.9, 0.999, 1e-8, 0.0).unwrap(),
MeanSquaredError::new(),
);
model.fit(&x, &y, 5).unwrap();
let preds = model.predict(&x).unwrap();
println!("prediction shape: {:?}", preds.shape());
}
```
[The Sequential Model](./3.1._The_Sequential_Model.md) is the container that turns a pile of layers into a trainable network. It owns the training loop, the optimizer, and the loss, and exposes `add`, `compile`, `fit`, `train_batch`, `evaluate`, `predict`, `summary`, and weight save/load. It also holds the batch-shuffle seed (`set_seed`) and the learning-rate pair (`learning_rate` / `set_learning_rate`). Read this section first, even if you need only one specific layer.
[Dense Layers and Activations](./3.2._Dense_Layers_and_Activations.md) covers the fully connected layer, the main layer of tabular models and the output stage of most networks. It also covers the `Activation` enum (`ReLU`, `Sigmoid`, `Tanh`, `Softmax`, `Linear`), which you fold into a layer or use as a standalone layer. Read this section second. Everything after it assumes you know how a `Dense` layer declares its `input_dim` and `units`.
[Loss Functions](./3.3._Loss_Functions.md) is the objective half of `compile`. It covers mean squared error and mean absolute error for regression, and binary, categorical, and sparse-categorical cross-entropy for classification. Read this section closely. Its averaging conventions differ on purpose: some average per element, others average per prediction site, and switching between them quietly rescales your effective learning rate. `CategoricalCrossEntropy` and `SparseCategoricalCrossEntropy` take a `from_logits` flag that changes whether you need a `Softmax` on the output. `BinaryCrossEntropy` has no such flag, and always expects a probability in `(0, 1)`.
[Optimizers](./3.4._Optimizers.md) is the update half of `compile`. It covers SGD with momentum, Adam, AdamW, RMSprop, and AdaGrad. This section also covers clip-by-global-norm (`global_clipnorm`), coupled versus decoupled weight decay, and mid-training learning-rate scheduling. Sections 3.1 through 3.4 together give you a complete, trainable feed-forward network.
The remaining sections add specialized layers. All of them plug into the same `Sequential`. [Convolutional Layers](./3.5._Convolutional_Layers.md) provides 1D/2D/3D convolution, plus depthwise and separable variants for spatial data. [Pooling Layers](./3.6._Pooling_Layers.md) provides parameter-free max and average downsampling, plus their global variants. [Recurrent Layers](./3.7._Recurrent_Layers.md) covers `SimpleRNN`, `LSTM`, and `GRU` for sequences. [Regularization and Normalization Layers](./3.8._Regularization_and_Normalization_Layers.md) covers dropout (including spatial dropout), Gaussian noise, and batch, layer, group, and instance normalization. These layers depend on mode: they behave differently in `fit` than in `predict`, and the model switches the mode for you.
[Saving and Loading Weights](./3.9._Saving_and_Loading_Weights.md) closes the chapter. `save_to_path` and `load_from_path` persist weights only, in postcard binary format. They do not persist the architecture. Rebuild the identical layer stack in code, then load the weights into it. Read this section once you have a model worth keeping. See [Model Persistence in Depth](../Chapter-07/7.2._Model_Persistence_in_Depth.md) for the format details and version caveats.