mathverse-ai 0.1.2

AI/ML mathematical primitives: tensors, activations, losses, optimizers, attention, autograd, layers, models
Documentation

MathVerse AI

Crates.io docs.rs License: MIT OR Apache-2.0 Rust: 1.87+

AI/ML mathematical primitives: tensors, activations, losses, optimizers, and attention mechanisms — all in pure Rust with zero external dependencies.


Features

  • N-dimensional tensor — Broadcasting, reshaping, element-wise math, matrix multiply
  • Activation functions — ReLU, GELU, Swish, Mish, Softmax, Sigmoid, Tanh, and more
  • Loss functions — MSE, MAE, Huber, Cross-Entropy, Binary Cross-Entropy, KL Divergence, Hinge
  • Evaluation metrics — Accuracy, Precision, Recall, F1, Confusion Matrix, ROC AUC, R²
  • Stateful optimizers — SGD (with momentum), Adam, AdamW
  • Learning rate schedulers — Constant, Step Decay, Cosine Annealing, Linear Warmup
  • Attention math — QKV projection, scaled dot-product attention, multi-head attention
  • Positional encodings — Sinusoidal (additive), rotary (RoPE)
  • Normalization — Layer, Batch, RMS normalization
  • Neural network layers — Linear, Dropout, Sequential, MLP, TransformerBlock

Module Overview

Module Purpose Key Types
tensor N-D tensor with row-major layout, broadcasting, math ops Tensor::zeros, matmul, add, layer_norm, broadcast_to
activations Element-wise activation functions and derivatives relu, gelu, sigmoid, softmax, swish, mish
losses Regression and classification loss functions mse, cross_entropy, binary_cross_entropy_with_logits, huber
metrics Model evaluation metrics accuracy, f1, confusion_matrix, roc_auc, r_squared
optimizers Stateful gradient-based optimizers Sgd, Adam, AdamW, LrScheduler
attention Transformer attention math scaled_dot_product_attention, multi_head_attention, apply_rope
layers Neural network layers Linear, LayerNorm, BatchNorm, Dropout
models Model architectures Sequential, MLP, TransformerBlock
autograd Automatic differentiation Backward pass computation
data Data loading utilities DataLoader, Batch, train_test_split

Installation

[dependencies]
mathverse-ai = { path = "crates/mathverse-ai" }

Quick Start

use mathverse_ai::{Tensor, sigmoid, softmax, mse, Adam};

fn main() {
    // Create tensors
    let logits = Tensor::new(&[1, 3], &[2.0, 1.0, 0.5]).unwrap();
    let probs = softmax(&logits, 1).unwrap();
    println!("Softmax: {:?}", probs.data);
    // Softmax: [0.5906, 0.2447, 0.1647]

    // MSE loss
    let pred = Tensor::new(&[3], &[1.0, 2.0, 3.0]).unwrap();
    let target = Tensor::new(&[3], &[1.0, 2.0, 5.0]).unwrap();
    let loss = mse(&pred, &target).unwrap();
    println!("MSE: {:.4}", loss);
    // MSE: 1.3333

    // Adam optimizer minimizing f(x) = x²
    let mut opt = Adam::new(0.1, 0.9, 0.999, 1e-8, 0.0);
    let mut x = [5.0];
    for _ in 0..100 {
        let g = [2.0 * x[0]];
        opt.step(&mut x, &g);
    }
    println!("x after 100 steps: {:.6}", x[0]);
    // x after 100 steps: 0.000001
}

Module Documentation

Tensor

The core N-dimensional array type. Supports broadcasting (NumPy-style trailing dimensions), element-wise arithmetic, matrix multiply, reductions, normalization, and shape manipulation.

use mathverse_ai::Tensor;

// Constructors
let t = Tensor::zeros(&[2, 3]);
let t = Tensor::arange(0.0, 6.0, 1.0).reshape(&[2, 3]).unwrap();

// Broadcasting
let a = Tensor::new(&[2, 1], &[1.0, 2.0]).unwrap();
let b = Tensor::new(&[1, 3], &[10.0, 20.0, 30.0]).unwrap();
let c = a.add(&b).unwrap(); // shape [2, 3]

// Matrix multiply
let a = Tensor::new(&[2, 3], &[1.0, 2.0, 3.0, 4.0, 5.0, 6.0]).unwrap();
let b = Tensor::new(&[3, 2], &[7.0, 8.0, 9.0, 10.0, 11.0, 12.0]).unwrap();
let c = a.matmul(&b).unwrap(); // shape [2, 2]

// Layer normalization
let t = Tensor::new(&[2, 3], &[1.0, 2.0, 3.0, 4.0, 5.0, 6.0]).unwrap();
let norm = t.layer_norm(1e-5); // each row has mean ≈ 0

Activations

All activation functions operate element-wise on tensors. Derivative functions (suffixed _grad) are provided for backpropagation.

use mathverse_ai::{Tensor, relu, gelu, sigmoid, softmax, swish};

let t = Tensor::new(&[5], &[-2.0, -1.0, 0.0, 1.0, 2.0]).unwrap();
let r = relu(&t);   // [0.0, 0.0, 0.0, 1.0, 2.0]
let s = sigmoid(&t); // [0.1192, 0.2689, 0.5, 0.7311, 0.8808]

// Softmax (probabilities sum to 1)
let logits = Tensor::new(&[3], &[1.0, 2.0, 3.0]).unwrap();
let probs = softmax(&logits, 0).unwrap();
// [0.0900, 0.2447, 0.6652]

Loss Functions

use mathverse_ai::{Tensor, mse, cross_entropy, binary_cross_entropy_with_logits};

// Regression: MSE
let pred = Tensor::new(&[3], &[1.0, 2.0, 3.0]).unwrap();
let target = Tensor::new(&[3], &[1.0, 2.0, 5.0]).unwrap();
println!("MSE: {:.4}", mse(&pred, &target).unwrap());

// Classification: Cross-Entropy
let logits = Tensor::new(&[2, 3], &[10.0, 1.0, 1.0, 1.0, 10.0, 1.0]).unwrap();
let targets = Tensor::new(&[2], &[0.0, 1.0]).unwrap();
println!("CE: {:.4}", cross_entropy(&logits, &targets).unwrap());

Optimizers

use mathverse_ai::{Adam, AdamW, Sgd, LrScheduler, Schedule};

// Adam optimizer
let mut opt = Adam::new(0.001, 0.9, 0.999, 1e-8, 0.0);
let mut params = vec![5.0, -3.0, 2.0];
for _ in 0..1000 {
    let grads: Vec<f64> = params.iter().map(|p| 2.0 * p).collect();
    opt.step(&mut params, &grads);
}
// params ≈ [0, 0, 0]

// Cosine annealing LR schedule
let mut scheduler = LrScheduler::new(
    Schedule::CosineAnnealing { t_max: 100, eta_min: 0.0001 }
);
for _ in 0..100 {
    let lr = scheduler.get_lr();
    scheduler.step();
}

Attention

Transformer attention mechanisms: QKV projection, scaled dot-product attention, multi-head attention, and positional encodings.

use mathverse_ai::{Tensor, multi_head_attention, sinusoidal_encoding, apply_rope};

// Scaled dot-product attention
let q = Tensor::new(&[1, 2, 4], &[1.0; 8]).unwrap();
let k = Tensor::new(&[1, 2, 4], &[1.0; 8]).unwrap();
let v = Tensor::new(&[1, 2, 4], &[1.0; 8]).unwrap();
let (out, weights) = mathverse_ai::scaled_dot_product_attention(&q, &k, &v, None, None).unwrap();

// Sinusoidal positional encoding
let enc = sinusoidal_encoding(10, 64, 10000);
// enc shape: [1, 10, 64]

// RoPE (Rotary Position Embeddings)
let x = Tensor::randn(&[1, 8, 64]);
let y = apply_rope(&x, 8, 10000.0);

Formulas:

  • Attention: Attention(Q,K,V) = softmax(QKᵀ / √dₖ) V
  • RoPE: θ_i = pos / base^(2i/d), apply 2D rotation to each pair

License

MIT OR Apache-2.0 — see LICENSE.