Expand description
Distributed Training
Support for multi-GPU training with data and model parallelism.
Structs§
- Data
Parallel - Data parallel training
- Distributed
Config - Distributed training configuration
- Distributed
Data Parallel - Distributed Data Parallel (DDP)
- Gradient
Accumulator - Gradient accumulation
- Model
Parallel - Model parallel training
- Pipeline
Parallel - Pipeline parallelism
Enums§
- Distributed
Backend - Distributed training backend