Batch normalization for images as described in Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Cthe size of the spatial dimension to reduce. For 3d tensors this is the 0th dimension. For 4d tensors, this is the 1st dimension.
BatchNorm2D supports the following cases (see sections below for more details):
- Training: ModuleMut and OwnedTape on the input tensor
- Inference: Module and NoneTape on the input tensor.
NOTE: ModuleMut/NoneTape, and Module/OwnedTape will fail to compile.
let bn: BatchNorm2D<3> = Default::default(); let _ = bn.forward(Tensor3D::<3, 2, 2>::zeros()); let _ = bn.forward(Tensor4D::<4, 3, 2, 2>::zeros());
- Running statistics: updated with momentum
- Normalization: calculated using batch stats
- Running statistics: not updated
- Normalization: calculated using running stats
Scale for affine transform. Defaults to 1.0
Bias for affine transform. Defaults to 0.0
Spatial mean that is updated during training. Defaults to 0.0
Spatial variance that is updated during training. Defaults to 1.0
Added to variance before taking sqrt for numerical stability. Defaults to 1e-5
Controls exponential moving average of running stats.Defaults to 0.1
running_stat * (1.0 - momentum) + stat * momentum.
Returns the argument unchanged.