Struct DistributedExecutor 

pub struct DistributedExecutor { /* private fields */ }

Distributed executor that coordinates multi-device execution.

Implementations

impl DistributedExecutor

pub fn new( config: DistributedConfig, backend: Arc<RwLock<dyn CommunicationBackend>>, ) -> Result<Self, ExecutorError>

Create a new distributed executor.

pub fn strategy(&self) -> ParallelismStrategy

Get the parallelism strategy.

pub fn rank(&self) -> usize

Get the rank of this process.

pub fn world_size(&self) -> usize

Get the world size.

pub fn barrier(&self) -> Result<(), ExecutorError>

Synchronize all processes.

pub fn data_parallel(&self) -> Option<&DataParallelCoordinator>

Get data parallel coordinator.

pub fn model_parallel(&self) -> Option<&ModelParallelCoordinator>

Get model parallel coordinator.

pub fn pipeline_parallel(&self) -> Option<&PipelineParallelCoordinator>

Get pipeline parallel coordinator.

Trait Implementations

impl Drop for DistributedExecutor

fn drop(&mut self)

Executes the destructor for this type.