Crate caffe2op_groupnorm

Structs

• GetGroupNormGradient
  | Warning: mu and rsig are for backward usage or | reference. They should NOT be used as forward | activations as they have no direct gradients | computed.
• GroupNormGradientOp

  ---

• GroupNormOp
  | Group Normalization (GN) operation: | https://arxiv.org/abs/1803.08494 |

Functions

• compute_gradient_fused_params
• compute_internal_gradients
  | Math: | Y = gamma * (X - mu) * rsig + beta | let s = gamma * rsig | let b = beta - gamma * mu * rsig | Y = s * X + b | let n = K * HxW | dL/dX = dL/dY * dY/dX = dL/dY * (d(s * X)/dX + db/dX) | d(s * X)/dX = s + X * ds/dX = s + gamma * X * drsig/dX | db/dX = -gamma * u * drsig/dX - gamma * rsig * dmu/dX | drsig/dX = -rsig^3 * (X - mu) / n | dmu/dX = 1 / n
• compute_internal_gradients_f32_nchw
• compute_internal_gradients_f32_nhwc
• gamma_beta_backward
• group_norm_backward_f32_nhwc
• group_norm_backward_i32_nchw