Expand description
Neural network layer module system built on coeus_autograd.
§Module trait
Module<T, B> is the core abstraction. Its forward
method returns a typed ModuleError so backend, input-contract, and
module-state failures cannot be reported as successful execution.
§Layer families
- Linear —
Linear, weight + optional bias, Xavier/Kaiming init viainit. - Convolution —
Conv1d,Conv2d,Conv3dwith stride/padding/dilation. - Normalization —
LayerNorm,RMSNorm,BatchNorm1d/2d/3d,GroupNorm,InstanceNorm1d,InstanceNorm2d,InstanceNorm3d. - Pooling —
MaxPool2d,AvgPool2d,MaxPool3d,AvgPool3d. - Attention —
MultiHeadAttention,ScaledDotProductAttentionwithCausalMask/NullMask. - Transformer —
TransformerEncoder,TransformerDecoder,FeedForwardblocks. - Positional —
SinusoidalEncoding,RotaryEmbedding. - Composites —
Sequential,StaticSeq,Dropout,Embedding,Softmax.
Re-exports§
pub use activation::celu;pub use activation::elu;pub use activation::gelu;pub use activation::gelu_tanh;pub use activation::glu;pub use activation::hardshrink;pub use activation::hardsigmoid;pub use activation::hardswish;pub use activation::hardtanh;pub use activation::leaky_relu;pub use activation::log_sigmoid;pub use activation::mish;pub use activation::prelu;pub use activation::relu;pub use activation::sigmoid;pub use activation::silu;pub use activation::softplus;pub use activation::softshrink;pub use activation::softsign;pub use activation::tanh;pub use activation::tanhshrink;pub use activation::threshold;pub use activation::Celu;pub use activation::CeluOp;pub use activation::GeLU;pub use activation::GeLUTanh;pub use activation::Hardshrink;pub use activation::HardshrinkOp;pub use activation::Hardsigmoid;pub use activation::HardsigmoidOp;pub use activation::Hardswish;pub use activation::HardswishOp;pub use activation::Hardtanh;pub use activation::HardtanhOp;pub use activation::LeakyReLU;pub use activation::LogSigmoid;pub use activation::Mish;pub use activation::PReLU;pub use activation::ReLU;pub use activation::SiLU;pub use activation::Sigmoid;pub use activation::Softplus;pub use activation::Softshrink;pub use activation::SoftshrinkOp;pub use activation::Softsign;pub use activation::SoftsignOp;pub use activation::Tanh;pub use activation::Tanhshrink;pub use activation::Threshold;pub use activation::ThresholdNode;pub use activation::ELU;pub use activation::GLU;pub use attention::multi_head_attention_cross;pub use attention::MhaProjectionParams;pub use attention::MultiHeadAttention;pub use attention::ScaledDotProductAttention;pub use bilinear::bilinear;pub use bilinear::Bilinear;pub use conv::Conv;pub use conv::Conv1d;pub use conv::Conv2d;pub use conv::Conv3d;pub use conv::ConvDim;pub use conv::ConvTranspose;pub use conv::ConvTranspose1d;pub use conv::ConvTranspose2d;pub use conv::ConvTranspose3d;pub use conv::DepthwiseConv3d;pub use conv::Dim1D;pub use conv::Dim2D;pub use conv::Dim3D;pub use conv::Fold1d;pub use conv::Fold2d;pub use conv::Unfold1d;pub use conv::Unfold2d;pub use dropout::Dropout;pub use embedding::Embedding;pub use embeddingbag::EmbeddingBag;pub use embeddingbag::EmbeddingBagMode;pub use init::kaiming_uniform;pub use init::xavier_uniform;pub use interpolate::interpolate_1d;pub use interpolate::interpolate_2d;pub use linear::Linear;pub use loss::bce_with_logits;pub use loss::binary_cross_entropy;pub use loss::cosine_embedding_loss;pub use loss::cosine_similarity;pub use loss::cross_entropy_loss;pub use loss::ctc_loss;pub use loss::gaussian_nll_loss;pub use loss::hinge_embedding_loss;pub use loss::huber_loss;pub use loss::kl_divergence;pub use loss::l1_loss;pub use loss::margin_ranking_loss;pub use loss::mse_loss;pub use loss::multi_label_soft_margin_loss;pub use loss::multi_margin;pub use loss::nanmean;pub use loss::nansum;pub use loss::nll_loss;pub use loss::pairwise_distance;pub use loss::poisson_nll;pub use loss::smooth_l1_loss;pub use loss::soft_margin;pub use loss::triplet_margin_loss;pub use loss::triplet_margin_with_distance_loss;pub use module::Module;pub use module::ModuleError;pub use module::ModuleResult;pub use module::ParameterLoadError;pub use normalization::group_norm;pub use normalization::layer_norm;pub use normalization::rms_norm;pub use normalization::BatchNorm1d;pub use normalization::BatchNorm2d;pub use normalization::BatchNorm3d;pub use normalization::GroupNorm;pub use normalization::InstanceNorm1d;pub use normalization::InstanceNorm2d;pub use normalization::InstanceNorm3d;pub use normalization::LayerNorm;pub use normalization::NormalizedShape;pub use normalization::RMSNorm;pub use pool::AdaptiveAvgPool1d;pub use pool::AdaptiveAvgPool2d;pub use pool::AdaptiveMaxPool1d;pub use pool::AdaptiveMaxPool2d;pub use pool::AvgPool1d;pub use pool::AvgPool2d;pub use pool::AvgPool3d;pub use pool::GlobalAvgPool1d;pub use pool::GlobalAvgPool2d;pub use pool::GlobalAvgPool3d;pub use pool::GlobalMaxPool2d;pub use pool::GlobalMaxPool3d;pub use pool::MaxPool1d;pub use pool::MaxPool2d;pub use pool::MaxPool3d;pub use positional::RotaryEmbedding;pub use positional::SinusoidalEncoding;pub use regularization::AlphaDropout;pub use regularization::FeatureAlphaDropout;pub use regularization::GaussianNoise;pub use regularization::LocalResponseNorm;pub use rnn::Bidirectional;pub use rnn::GRUCell;pub use rnn::Gru;pub use rnn::LSTMCell;pub use rnn::Lstm;pub use rnn::RNNCell;pub use rnn::Rnn;pub use rnn::RnnNonlinearity;pub use sequential::ModuleExt;pub use sequential::Sequential;pub use sequential::StaticSeq;pub use softmax::softmax;pub use softmax::Softmax;pub use swiglu::SwiGlu;pub use transformer::feed_forward;pub use transformer::transformer_decoder_layer;pub use transformer::transformer_encoder_layer;pub use transformer::FeedForward;pub use transformer::Transformer;pub use transformer::TransformerDecoder;pub use transformer::TransformerDecoderLayer;pub use transformer::TransformerDecoderLayerParams;pub use transformer::TransformerEncoder;pub use transformer::TransformerEncoderLayer;pub use transformer::TransformerEncoderLayerParams;
Modules§
- activation
- Activation functions (ReLU, GeLU, SiLU, etc.).
- attention
- Attention mechanisms (MHA, SDP, masks).
- bilinear
- Bilinear interaction layer.
- conv
- Convolution layers (1D/2D/3D and transposed variants).
- dropout
- Dropout regularization layer.
- embedding
- Embedding lookup layer.
- embeddingbag
- EmbeddingBag aggregation layer.
- init
- Weight initialization utilities (Xavier, Kaiming). Provider-dispatched neural-network parameter initialization.
- interpolate
- Spatial interpolation operations.
- linear
- Fully-connected linear layer.
- loss
- Loss functions (cross-entropy, MSE, etc.). Loss functions for classification, regression, and sequence alignment.
- module
- Core
Moduletrait for neural network layers. Neural-network module contracts and typed failures. - normalization
- Normalization layers (LayerNorm, BatchNorm, GroupNorm, etc.).
- pool
- Learnable parameter wrapper. Pooling layers (max, average, global).
- positional
- Positional encoding layers (sinusoidal, RoPE).
- regularization
- Regularization layers (AlphaDropout, FeatureAlphaDropout, GaussianNoise, LocalResponseNorm).
- rnn
- Recurrent layers (LSTM, GRU).
- sequential
- Sequential and static-sequence module containers.
- softmax
- Softmax layer and functional softmax.
- swiglu
- SwiGLU gated feed-forward unit. SwiGLU gated feed-forward unit.
- transformer
- Transformer encoder, decoder, and sub-layers.
Structs§
- Causal
Mask - Differentiable operations.
opscurates its own public surface, so the crate root re-exports it wholesale rather than repeating the 188-name list. Causal (lower-triangular) attention mask — positions j > i are masked to −∞. - Null
Mask - Differentiable operations.
opscurates its own public surface, so the crate root re-exports it wholesale rather than repeating the 188-name list. No masking (full attention).
Enums§
- Resize
Mode - Resize mode.
Traits§
- Attention
Mask - Differentiable operations.
opscurates its own public surface, so the crate root re-exports it wholesale rather than repeating the 188-name list. ZST marker trait for attention mask strategies.