Module dfdx::nn

Expand description

High level neural network building blocks such as Linear, activations, and tuples as Modules. Also includes .save() & .load() for all Modules.

Saving and loading model parameters is done using SaveToNpz and LoadFromNpz. All modules provided here implement it, including tuples. So you can call SaveToNpz::save() to save the module to a .npz zip file format, and then LoadFromNpz::load() to load the weights.

Randomizing parameters is done using ResetParams::reset_params(). All modules implement the underlying logic themselves. For example Linear calculates the distribution it draws values from based on its input size.

Structs

Abs

Unit struct that impls Module as calling abs() on input.

Cos

Unit struct that impls Module as calling cos() on input.

Dropout

A Module that calls dropout() in Module::forward() with probability self.p.

DropoutOneIn

A Module that calls dropout() in Module::forward() with probability 1.0 / N. Note that dropout() does not do anything for tensors with NoTape.

Exp

Unit struct that impls Module as calling exp() on input.

LayerNorm1D

Implements layer normalization as described in Layer Normalization.

Linear

A linear transformation of the form x * transpose(W) + b, where W is a matrix, x is a vector or matrix, and b is a vector. If x is a matrix this does matrix multiplication.

Ln

Unit struct that impls Module as calling ln() on input.

ReLU

Unit struct that impls Module as calling relu() on input.

Repeated

Repeats T N times. This requires that T’s input is the same as it’s output.

Residual

Represents a residual connection around F: F(x) + x, as introduced in Deep Residual Learning for Image Recognition.

Sigmoid

Unit struct that impls Module as calling sigmoid() on input.

Sin

Unit struct that impls Module as calling sin() on input.