Nabla ML

A lightweight neural network library implemented in Rust, focusing on simplicity and educational purposes. It is inspired by NumPy, providing a multi-dimensional array implementation with various mathematical and array manipulation functionalities.

Features

Core Components

NDArray: A multi-dimensional array implementation supporting:
- Basic operations (add, subtract, multiply, divide)
- Broadcasting support for operations like [N, M] + [1, M]
- Shape manipulation (reshape, transpose)
- Matrix operations (dot product)
- Statistical operations (sum, mean)
- Element-wise operations (exp, log, sqrt)
- Padding functionality for batches
Moreover...
- Array Creation: Create 1D and 2D arrays using vectors and matrices.
- Random Arrays: Generate arrays with random numbers, including uniform and normal distributions.
- Arithmetic Operations: Perform element-wise addition, subtraction, multiplication, and division.
- Mathematical Functions: Apply functions like square root, exponential, sine, cosine, logarithm, hyperbolic tangent, ReLU, Leaky ReLU, and Sigmoid to arrays.
- Array Reshaping: Change the shape of arrays while maintaining data integrity.
- File I/O: Save and load arrays in a compressed format.
- Linear Regression: Perform linear regression using gradient descent.
- MNIST Dataset Handling: Convert and load MNIST data for machine learning tasks.
Neural Network Layers:
- Dense (Fully Connected) layers
- Activation layers
- Support for various activation functions

Activation Functions

ReLU (Rectified Linear Unit)
Leaky ReLU
Sigmoid
Softmax (for classification)

Optimizers

Adam optimizer with configurable parameters:
- Learning rate
- Beta1 and Beta2 momentum parameters
- Epsilon for numerical stability
- Automatic moment vector shape handling

Training Features

Mini-batch training support
Automatic batch padding
Loss tracking
Accuracy metrics
Broadcasting for efficient computations

Example Usage

let mut model = Model::new()
    .input(vec![784])  // For MNIST: 28x28 = 784 input features
    .add_dense(32, Box::new(Sigmoid::default()))
    .add_dense(32, Box::new(Sigmoid::default()))
    .add_dense(10, Box::new(Softmax::default()))
    .build();

model.compile(
    Box::new(Adam::default()), 
    Box::new(CategoricalCrossentropy),
    vec!["accuracy".to_string()]
);

// Train the model
let history = model.fit(
    &training_data, 
    &training_labels, 
    batch_size: 32, 
    epochs: 10
);

model.summary();

Usage

Array Creation

use nabla_ml::NDArray;

let arr = NDArray::from_vec(vec![1.0, 2.0, 3.0]);
let matrix = NDArray::from_matrix(vec![
    vec![1.0, 2.0, 3.0],
    vec![4.0, 5.0, 6.0],
]);

Random Arrays

use nabla_ml::NDArray;

let random_array = NDArray::randn(5);
let random_matrix = NDArray::randn_2d(3, 3);

Mathematical Functions

use nabla_ml::NDArray;

let arr = NDArray::from_vec(vec![0.0, 1.0, -1.0]);
let sqrt_arr = arr.sqrt();
let exp_arr = arr.exp();
let tanh_arr = arr.tanh();
let relu_arr = arr.relu();
let leaky_relu_arr = arr.leaky_relu(0.01);
let sigmoid_arr = arr.sigmoid();

File I/O with .nab Format

use nabla_ml::{NDArray, save_nab, load_nab};

let array = NDArray::from_vec(vec![1.0, 2.0, 3.0, 4.0]);
save_nab("data.nab", &array).expect("Failed to save array");

let loaded_array = load_nab("data.nab").expect("Failed to load array");
assert_eq!(array.data(), loaded_array.data());
assert_eq!(array.shape(), loaded_array.shape());

Linear Regression

use nabla_ml::NDArray;

let X = NDArray::from_matrix(vec![
    vec![0.0, 0.0],
    vec![1.0, 0.0],
    vec![0.0, 1.0],
    vec![1.0, 1.0],
]);
let y = NDArray::from_vec(vec![1.0, 2.0, 3.0, 4.0]);

let (theta, history) = NDArray::linear_regression(&X, &y, 0.01, 1000);
println!("Parameters: {:?}", theta);

MNIST Dataset Handling

use nabla_ml::NDArray;

NDArray::mnist_csv_to_nab(
    "csv/mnist_test.csv",
    "datasets/mnist_test_images.nab",
    "datasets/mnist_test_labels.nab",
    vec![28, 28]
).expect("Failed to convert MNIST CSV to NAB format");

let ((train_images, train_labels), (test_images, test_labels)) = 
    NDArray::load_and_split_dataset("datasets/mnist_test", 80.0).expect("Failed to load and split dataset");

Mnist dataset in .nab format can be found here

One-Hot Encoding

use nabla_ml::NDArray;

// Convert labels to one-hot encoded format
let labels = NDArray::from_vec(vec![-1.0, 0.0, 1.0, 0.0, -1.0]);
let one_hot = NDArray::one_hot_encode(&labels);

// Result will be a 2D NDArray:
// [1, 0, 0]  # Class 0
// [0, 1, 0]  # Class 1
// [0, 0, 1]  # Class 2
// [0, 1, 0]  # Class 1
// [1, 0, 0]  # Class 0