Struct DistributedTrainer 

pub struct DistributedTrainer { /* private fields */ }

Distributed trainer for multi-GPU/multi-node training

Implementations

impl DistributedTrainer

pub fn new( config: TrainingConfig, distributed_config: DistributedConfig, ) -> Self

pub fn with_optimizer(self, optimizer: OptimizerType) -> Self

pub fn with_scheduler(self, scheduler: LearningRateScheduler) -> Self

pub async fn train_distributed( &mut self, model: Arc<RwLock<dyn EmbeddingModel + Send + Sync>>, ) -> Result<TrainingStats>

Run the multi-worker training loop over a shared model.

Execution model (read before relying on this for scaling): every worker operates on the same Arc<RwLock<dyn EmbeddingModel>> and replicates the training loop, while the coordinator performs a real synchronous all-reduce (sum-then-average across world_size ranks) of the per-rank loss statistics exchanged each sync round and broadcasts the averaged result back. Because the model is shared rather than sharded, this is a training-loop-replication + loss-all-reduce harness, not full parameter-level data-parallel SGD (no per-rank parameter gradients are exchanged — there is a single shared parameter set). The all-reduce path itself is genuinely executed, not simulated.