Enum ReduceMode 

pub enum ReduceMode {
    Ring,
    Deterministic,
}

Reduction algorithm + precision for ProcessGroup::all_reduce. Selected per call, or globally via env (see env_reduce_mode).

Variants

Ring

Bandwidth-optimal ring, f32 accumulation. Fastest; f32 precision, and the last-ulp result depends on world size. Reproducible run-to-run. Default.

Deterministic

Deterministic + precise, at ring speed. The same bandwidth-optimal ring, but its reduce-scatter accumulates in f64 (the all-gather stays f32 — those values are already reduced, so there is no precision left to lose). The fold order is fixed by the algorithm, so the result is bitwise reproducible run-to-run; f64 accumulation makes each element the correctly-rounded exact cross-rank sum, so it neither loses precision nor biases the gradient and is effectively independent of world size. Only the reduce-scatter carries an f64 payload (≈1.5× the f32 ring’s bytes) — NOT a gather-to-root’s O(n·len), so speed is preserved. The mode to pick when you want reproducible, precise data-parallel training.

Trait Implementations

impl Clone for ReduceMode

fn clone(&self) -> ReduceMode

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for ReduceMode

impl Debug for ReduceMode

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl Eq for ReduceMode

impl PartialEq for ReduceMode

fn eq(&self, other: &ReduceMode) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for ReduceMode