Struct DataParallelCoordinator 

pub struct DataParallelCoordinator { /* private fields */ }

Coordinates data-parallel training across multiple GPUs.

Each GPU holds a complete replica of the model. Per training step:

1. Split mini-batch into N shards (one per GPU)
2. Each GPU processes its shard independently (via std::thread::scope)
3. Average LoRA gradients on CPU (they’re CPU-resident Tensor values)
4. Apply optimizer step on all replicas

Implementations

impl DataParallelCoordinator

pub fn new( model_config: &TransformerConfig, classify_config: ClassifyConfig, gpu_indices: &[u32], ) -> Result<Self, String>

Create a data-parallel coordinator with the given GPU indices.

Creates one ClassifyPipeline per GPU, each with its own WgpuForwardPass targeting a specific adapter.

Arguments

• model_config - Transformer architecture configuration
• classify_config - Classification training configuration
• gpu_indices - wgpu adapter indices to use (e.g., [0, 1])

Errors

Returns error if any GPU pipeline creation fails.

pub fn num_gpus(&self) -> usize

Number of GPUs in the pool

pub fn primary_pipeline(&mut self) -> &mut ClassifyPipeline

Get a mutable reference to the first pipeline (for evaluation/inference)

pub fn primary_pipeline_ref(&self) -> &ClassifyPipeline

Get an immutable reference to the first pipeline

pub fn train_batch_parallel(&mut self, samples: &[SafetySample]) -> BatchResult

Train one batch across all GPUs in parallel.

Splits samples across GPUs, runs forward/backward in parallel threads, averages LoRA gradients, and applies the optimizer step.

Contract (C-DP-002)

• Precondition: samples.len() >= num_gpus for balanced sharding
• Postcondition: All pipelines have updated, identical LoRA weights
• Invariant: Gradient averaging preserves numerical stability