Enum AllReduceStrategy 

pub enum AllReduceStrategy {
    Centralized,
    Tree(u32),
    Ring,
}

All reduce can be implemented with different algorithms, which all have the same result.

Variants

Centralized

One device is the “central”. The other devices, “peripherals”, send their tensors to the central. The central does the reduction, and sends the result back to each peripheral.

Tree(u32)

Devices are organized in a tree structure (with a given arity). Each node reduces its children’s tensors with its own, and sends the result to its parent. Leaf nodes will simply send their tensors to their parents. When the root node calculates the result, it is propagated down the tree.

Ring

Devices are organized in a ring. The tensors are split into N slices, where N is the number of devices participating. The slices are progressively sent around the ring until every device has one fully reduced slice of the tensor. Then, the resulting slices are sent around until every device has the full result. See ring.rs for details.

Trait Implementations

impl Clone for AllReduceStrategy

fn clone(&self) -> AllReduceStrategy

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for AllReduceStrategy

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for AllReduceStrategy

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl PartialEq for AllReduceStrategy

fn eq(&self, other: &AllReduceStrategy) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Serialize for AllReduceStrategy

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl Copy for AllReduceStrategy

impl Eq for AllReduceStrategy

impl StructuralPartialEq for AllReduceStrategy