Skip to main content

sum_gradients

Function sum_gradients 

Source
pub fn sum_gradients(x: &Array, group: Option<&Group>) -> Array
Expand description

x unchanged, whose gradient is summed over group (MLX’s sum_gradients): the input of a layer whose output is sharded across the group, so each rank’s gradient covers every rank’s slice of the output.