rustyasg 0.4.1

Graph-based deep learning framework in Rust: define-then-run ASG, graph-to-graph autograd, wgpu GPU backend, and an interactive egui graph visualizer.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107

//! # Neural Network Layers Module
//!
//! This module contains building blocks for constructing neural networks.
//!
//! In the graph-based architecture, each "layer" is a constructor that adds
//! a specific pattern of nodes (operations and parameters) to the ASG.
//!
//! ## Available Layers
//!
//! ### Core Layers
//! - [`Linear`]: Fully connected / dense layer
//! - [`Conv2d`]: 2D convolution with configurable stride, padding, dilation
//! - [`ConvTranspose2d`]: Transposed 2D convolution (deconvolution)
//! - [`Embedding`]: Embedding lookup table
//!
//! ### Normalization
//! - [`LayerNorm`]: Layer normalization
//! - [`BatchNorm`]: Batch normalization
//!
//! ### Activations
//! - [`ReLU`], [`LeakyReLU`], [`ELU`]: Rectified linear units
//! - [`Sigmoid`], [`Tanh`]: Classic activations
//! - [`GELU`], [`SiLU`]/[`Swish`]: Modern smooth activations
//! - [`Softmax`], [`Softplus`]: Output activations
//!
//! ### Attention & Transformers
//! - [`MultiHeadAttention`]: Multi-head self/cross attention
//! - [`TransformerBlock`]: Complete transformer encoder block
//! - [`FeedForward`]: Position-wise feed-forward network
//!
//! ### Positional Encodings
//! - [`SinusoidalPositionalEncoding`]: Fixed sinusoidal encoding
//! - [`LearnedPositionalEmbedding`]: Learnable position embeddings
//! - [`RotaryPositionEmbedding`]: RoPE, used in LLaMA and similar
//! - [`ALiBi`]: Attention with Linear Biases
//!
//! ### Pooling
//! - [`MaxPool2d`], [`AvgPool2d`]: Standard pooling
//! - [`GlobalAvgPool2d`], [`AdaptiveAvgPool2d`]: Global/adaptive pooling
//!
//! ### Regularization
//! - [`Dropout`]: Standard dropout
//! - [`SpatialDropout`]: Dropout for convolutional features
//!
//! ## Example
//!
//! ```ignore
//! use rustyasg::nn::{Linear, ReLU, Module};
//! use rustyasg::tensor::{GraphContext, Tensor};
//!
//! let ctx = Rc::new(RefCell::new(GraphContext::new()));
//! let linear = Linear::new(&ctx, "fc1", 784, 128);
//! let relu = ReLU;
//!
//! let x = Tensor::new_input(&ctx, "input");
//! let h = relu.forward(&linear.forward(&x));
//! ```

// Declare all submodules
pub mod activations;
pub mod attention;
pub mod batchnorm;
pub mod conv;
pub mod dropout;
pub mod embedding;
pub mod feedforward;
pub mod init;
pub mod linear;
pub mod module;
pub mod norm;
pub mod pooling;
pub mod positional;
pub mod transformer;

// Re-export structures for convenience

// Activations
pub use activations::{LeakyReLU, ReLU, SiLU, Sigmoid, Softmax, Softplus, Swish, Tanh, ELU, GELU};

// Convolutional layers
pub use conv::{Conv2d, Conv2dConfig, ConvTranspose2d};

// Pooling layers
pub use pooling::{AdaptiveAvgPool2d, AvgPool2d, GlobalAvgPool2d, MaxPool2d};

// Other layers
pub use attention::{
    create_causal_mask, create_padding_mask_from_ids, AttentionMask, MultiHeadAttention,
    MultiHeadAttentionConfig,
};
pub use batchnorm::BatchNorm;
pub use dropout::{Dropout, SpatialDropout};
pub use embedding::Embedding;
pub use feedforward::FeedForward;
pub use linear::Linear;
pub use norm::LayerNorm;
pub use positional::{
    create_position_ids, ALiBi, LearnedPositionalEmbedding, RotaryPositionEmbedding,
    SinusoidalPositionalEncoding,
};
pub use transformer::TransformerBlock;

// Base trait
pub use module::Module;

// Initializers
pub use init::Initializer;