Expand description
Single-process, multi-device collectives — lazy-only.
§Why this exists alongside crate::comm::Communicator
Communicator is the SPMD shape: each rank calls all_reduce with its
own tensor and the implementation rendezvouses with the other ranks. That is
the right shape for multi-process / multi-node.
A single process driving N devices holds every shard at once. There is no
other rank to meet, so the per-rank signature does not fit — which is why
IdentityComm (the only Communicator impl today) returns its input: with
world_size == 1 there is genuinely nothing to reduce. That is correct for a
single rank and is not a working collective.
DeviceGroup is the slice-shaped counterpart: you hand it all the shards.
“One process, four GPUs” — the configuration most inference deployments
actually run — needs cross-device copies, not a multi-node transport.
§Why this module is lazy-only
Everything tensor-touching in this crate targets Tensor, and every
model in Fuel is built on the lazy graph. A collective written against the
old eager Tensor could not have reduced lazy shards, which is the likeliest
reason this crate has no consumers. B6 deleted that eager Tensor outright
(fuel-core/src/tensor.rs is gone), so lazy-only is now the only option
rather than a preference.
§Transport, and why cross-vendor hops are authored here
Movement is Tensor::copy_to_device → Op::Copy, resolved by the
executor’s Copy arm. Every hop with CPU on one end is a single Op::Copy
— CPU↔CUDA, CPU↔Vulkan, and same-device. A direct cross-vendor GPU↔GPU copy
is not implemented: both GPU wrappers reject a foreign-GPU output, and
fuel-graph/src/opt.rs:2590 says so outright — “a cross-VENDOR GPU↔GPU edge
(e.g. CUDA↔Vulkan) can’t be done as one Op::Copy”.
The optimizer does split such an edge into two hops, but
insert_cross_device_copies deliberately skips any consumer that is itself
an Op::Copy/Op::Move (opt.rs:2571-2577) — otherwise inserting a copy on
a copy’s input would regress infinitely. That carve-out is correct, and it
means a user-authored copy across vendors is never split for you.
So this module stages through CPU itself when source and leader are
different non-CPU backends: src → CPU → leader, two explicit Op::Copy
nodes. A group may therefore genuinely span CPU + CUDA + Vulkan.
(An earlier version of this comment claimed the path was “backend-agnostic …
without special-casing”. That was inferred from copy_to_device’s doc comment
rather than read from the pass, and it was wrong.)
Structs§
- Device
Group - A set of devices participating in single-process collectives.