Module ops

Module ops 

Source
Expand description

Basic neural network operations

Structs§

Embedding
Embedding layer
LayerNorm
LayerNorm
Linear
Linear layer (fully connected)
RMSNorm
RMSNorm (Root Mean Square Layer Normalization)

Functions§

gelu
GELU activation
relu
ReLU activation
silu
SiLU (Swish) activation function