Module mpi_collectives

Source

Modules§

mpi_request
Nested message and enum types in MPIRequest.
mpi_response
Nested message and enum types in MPIResponse.

Structs§

MpiRequest
An MPIRequest is a message sent from a rank greater than zero to the coordinator (rank zero), informing the coordinator of an operation that the rank wants to do and the tensor that it wants to apply the operation to.
MpiResponse
An MPIResponse is a message sent from the coordinator (rank zero) to a rank greater than zero, informing the rank of an operation should be performed now. If the operation requested would result in an error (for example, due to a type or shape mismatch), then the MPIResponse can contain an error and an error message instead. Finally, an MPIResponse can be a DONE message (if there are no more tensors to reduce on this tick of the background loop) or SHUTDOWN if all MPI processes should shut down.