Expand description
AI/ML mathematical primitives: tensors, activations, losses, optimizers, attention, layers, models.
§Features
- N-dimensional tensor with broadcasting, reshaping, and element-wise math
- Activation functions: ReLU, GELU, Swish, Mish, Softmax, Sigmoid, Tanh, and more
- Loss functions: MSE, MAE, Huber, Cross-Entropy, Binary Cross-Entropy, KL Divergence, Hinge
- Evaluation metrics: Accuracy, Precision, Recall, F1, Confusion Matrix, ROC AUC, R², Explained Variance
- Stateful optimizers: SGD (with momentum), Adam, AdamW
- Learning rate schedulers: Constant, Step Decay, Cosine Annealing, Linear Warmup
- Attention math: QKV projection, scaled dot-product attention, multi-head attention
- Positional encodings: sinusoidal (additive), rotary (RoPE)
- Layer/Batch/RMS normalization
Re-exports§
pub use tensor::Tensor;pub use activations::relu;pub use activations::sigmoid;pub use activations::softmax;pub use activations::gelu;pub use activations::swish;pub use activations::tanh as tanh_act;pub use losses::mse;pub use losses::cross_entropy;pub use losses::binary_cross_entropy_with_logits;pub use metrics::accuracy;pub use metrics::f1;pub use metrics::confusion_matrix;pub use optimizers::Sgd;pub use optimizers::Adam;pub use optimizers::AdamW;pub use optimizers::LrScheduler;pub use optimizers::Schedule;pub use attention::scaled_dot_product_attention;pub use attention::multi_head_attention;pub use attention::sinusoidal_encoding;pub use attention::apply_rope;pub use layers::Linear;pub use layers::LayerNorm;pub use layers::BatchNorm;pub use layers::Dropout;pub use models::Sequential;pub use models::MLP;pub use models::TransformerBlock;pub use models::Activation;pub use data::DataLoader;pub use data::Batch;pub use data::train_test_split;
Modules§
- activations
- Activation functions: element-wise and softmax variants with derivatives.
- attention
- Attention math: QKV projection, scaled dot-product attention, multi-head attention, sinusoidal positional encoding, rotary embeddings (RoPE).
- attention_
adv - Advanced attention mechanisms: flash, grouped query, cross, and linear attention.
- autograd
- Automatic differentiation via reverse-mode autodiff (backpropagation).
- conv_
transpose - Transposed convolution (deconvolution) for 1D and 2D tensors.
- data
- Data loading utilities: DataLoader with batching and shuffling.
- generative
- Generative model components: VAE, GAN, and diffusion utilities.
- layers
- Neural network layers: Linear, Conv1d, Conv2d, LayerNorm, BatchNorm, Dropout.
- losses
- Loss functions: regression (MSE, MAE, Huber) and classification (cross-entropy, etc.).
- metrics
- Evaluation metrics: classification (accuracy, precision, recall, F1, ROC AUC) and regression (MSE, MAE, R², explained variance).
- models
- Sequential model container: chains layers + activations for forward pass.
- optimizers
- Stateful optimizers: SGD (with momentum + weight decay), Adam, AdamW, and learning rate schedulers.
- recurrent
- Recurrent layers: vanilla RNN, LSTM, and GRU forward passes.
- registry
- Model registry: save/load tensors to text files, parameter counting.
- tensor
- N-dimensional tensor with row-major layout, broadcasting, and math ops.
- vision
- Vision Transformer (ViT) components: patch embedding, CLS token, positional embeddings.