1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
//! Coordinate CPU training processes and save portable distributed state.
//!
//! [`ProcessGroup`] provides blocking TCP collectives on dense CPU `Float`,
//! `Double`, and `Int64` tensors. [`DistributedDataParallel`] synchronizes an
//! ordinary model explicitly; [`ShardedTrainer`] keeps parameter and optimizer
//! shards between functional training steps. Neither requires Python at runtime.
//!
//! Connections are unencrypted and intended for mutually trusted workers. Use
//! an isolated network, a unique session name, identical operation order, and a
//! finite timeout. A failed communication poisons the group; restart all ranks
//! from their last completed checkpoint instead of retrying a partial step.
//!
//! ```
//! use rusttorch::{Tensor, distributed::{ProcessGroup, GroupOptions, ReduceOp}};
//! use std::net::TcpListener;
//! // A one-process group is useful for testing the same training program locally.
//! let listener = TcpListener::bind("127.0.0.1:0").unwrap();
//! let mut group = ProcessGroup::accept(listener, 1, GroupOptions::new("example"))?;
//! let total = group.all_reduce(&Tensor::from_slice(&[2_f64, 3.]), ReduceOp::Sum)?;
//! assert_eq!(Vec::<f64>::try_from(total)?, [2., 3.]);
//! # Ok::<(), rusttorch::RustTorchError>(())
//! ```
//!
//! See the [distributed training guide](crate::tutorials) for complete launch,
//! accumulation, checkpoint and resharding workflows. CUDA/NCCL, asynchronous
//! work handles, automatic backward hooks and communication overlap are absent.
pub use ;
pub use ;
use crate::;