MathVerse AI
AI/ML mathematical primitives: tensors, activations, losses, optimizers, and attention mechanisms — all in pure Rust with zero external dependencies.
Features
- N-dimensional tensor — Broadcasting, reshaping, element-wise math, matrix multiply
- Activation functions — ReLU, GELU, Swish, Mish, Softmax, Sigmoid, Tanh, and more
- Loss functions — MSE, MAE, Huber, Cross-Entropy, Binary Cross-Entropy, KL Divergence, Hinge
- Evaluation metrics — Accuracy, Precision, Recall, F1, Confusion Matrix, ROC AUC, R²
- Stateful optimizers — SGD (with momentum), Adam, AdamW
- Learning rate schedulers — Constant, Step Decay, Cosine Annealing, Linear Warmup
- Attention math — QKV projection, scaled dot-product attention, multi-head attention
- Positional encodings — Sinusoidal (additive), rotary (RoPE)
- Normalization — Layer, Batch, RMS normalization
- Neural network layers — Linear, Dropout, Sequential, MLP, TransformerBlock
Module Overview
| Module | Purpose | Key Types |
|---|---|---|
tensor |
N-D tensor with row-major layout, broadcasting, math ops | Tensor::zeros, matmul, add, layer_norm, broadcast_to |
activations |
Element-wise activation functions and derivatives | relu, gelu, sigmoid, softmax, swish, mish |
losses |
Regression and classification loss functions | mse, cross_entropy, binary_cross_entropy_with_logits, huber |
metrics |
Model evaluation metrics | accuracy, f1, confusion_matrix, roc_auc, r_squared |
optimizers |
Stateful gradient-based optimizers | Sgd, Adam, AdamW, LrScheduler |
attention |
Transformer attention math | scaled_dot_product_attention, multi_head_attention, apply_rope |
layers |
Neural network layers | Linear, LayerNorm, BatchNorm, Dropout |
models |
Model architectures | Sequential, MLP, TransformerBlock |
autograd |
Automatic differentiation | Backward pass computation |
data |
Data loading utilities | DataLoader, Batch, train_test_split |
Installation
[]
= { = "crates/mathverse-ai" }
Quick Start
use ;
Module Documentation
Tensor
The core N-dimensional array type. Supports broadcasting (NumPy-style trailing dimensions), element-wise arithmetic, matrix multiply, reductions, normalization, and shape manipulation.
use Tensor;
// Constructors
let t = zeros;
let t = arange.reshape.unwrap;
// Broadcasting
let a = new.unwrap;
let b = new.unwrap;
let c = a.add.unwrap; // shape [2, 3]
// Matrix multiply
let a = new.unwrap;
let b = new.unwrap;
let c = a.matmul.unwrap; // shape [2, 2]
// Layer normalization
let t = new.unwrap;
let norm = t.layer_norm; // each row has mean ≈ 0
Activations
All activation functions operate element-wise on tensors. Derivative functions (suffixed _grad) are provided for backpropagation.
use ;
let t = new.unwrap;
let r = relu; // [0.0, 0.0, 0.0, 1.0, 2.0]
let s = sigmoid; // [0.1192, 0.2689, 0.5, 0.7311, 0.8808]
// Softmax (probabilities sum to 1)
let logits = new.unwrap;
let probs = softmax.unwrap;
// [0.0900, 0.2447, 0.6652]
Loss Functions
use ;
// Regression: MSE
let pred = new.unwrap;
let target = new.unwrap;
println!;
// Classification: Cross-Entropy
let logits = new.unwrap;
let targets = new.unwrap;
println!;
Optimizers
use ;
// Adam optimizer
let mut opt = new;
let mut params = vec!;
for _ in 0..1000
// params ≈ [0, 0, 0]
// Cosine annealing LR schedule
let mut scheduler = new;
for _ in 0..100
Attention
Transformer attention mechanisms: QKV projection, scaled dot-product attention, multi-head attention, and positional encodings.
use ;
// Scaled dot-product attention
let q = new.unwrap;
let k = new.unwrap;
let v = new.unwrap;
let = scaled_dot_product_attention.unwrap;
// Sinusoidal positional encoding
let enc = sinusoidal_encoding;
// enc shape: [1, 10, 64]
// RoPE (Rotary Position Embeddings)
let x = randn;
let y = apply_rope;
Formulas:
- Attention:
Attention(Q,K,V) = softmax(QKᵀ / √dₖ) V - RoPE:
θ_i = pos / base^(2i/d), apply 2D rotation to each pair
License
MIT OR Apache-2.0 — see LICENSE.