Module distributed_gpu 

Distributed GPU optimization combining MPI and GPU acceleration

This module provides optimization algorithms that leverage both distributed computing (MPI) and GPU acceleration, enabling massive parallel optimization across multiple nodes with GPU acceleration on each node.

GPU function-evaluation interface (gpu feature)

When the gpu feature is enabled (scirs2-core/array_protocol_wgpu) and a wgpu adapter is available at runtime, the differential-evolution inner loop offloads its parallelizable numeric kernels to the GPU through the scirs2_core::array_protocol::gpu_ndarray::GpuNdarray dispatch surface, mirroring the canonical unconstrained::lbfgs_gpu implementation:

• Batch fitness reduction (evaluate_population_gpu) — the per-row objective values are computed by the caller-supplied closure (an opaque Fn, so it cannot itself run on-device), but the population’s sum-of-fitness reduction used by convergence/aggregation is computed on the GPU via GpuNdarray::sum_all. The on-device aggregate is validated against the CPU sum within an f32 tolerance and is otherwise discarded — the returned fitness is always the exact CPU result.
• Mutation / crossover donor math (gpu_mutation_crossover) — the differential-evolution donor vector v = a + F · (b − c) is evaluated for the whole population as flat GPU array arithmetic (subtract, multiply_by_scalar_f32, add); the binomial crossover mask is applied on the host.
• Selection (gpu_selection) — the elementwise greedy replacement where(trial ≤ current) is staged through a GPU subtract reduction that produces the per-individual fitness deltas used to drive the mask.

Every GPU path is gated (problem size ≥ GPU_DISTRIBUTED_THRESHOLD), probed (adapter availability is cached in a OnceLock), and fail-safe (any upload/dispatch/download error, or a missing adapter, transparently falls back to the CPU implementation — the public interface and numerical semantics are identical with or without the GPU).

Precision note

GpuNdarray operates on f32; values are cast on upload and back to f64 on download. GPU and CPU results therefore agree only to single precision (~1e-3 relative). Hosts requiring full f64 accuracy should build without the gpu feature, which compiles the CPU path unconditionally.

Structs

DistributedGpuConfig

Configuration for distributed GPU optimization

DistributedGpuOptimizer

Distributed GPU optimization context

DistributedGpuResults

Results from distributed GPU optimization

DistributedGpuStats

Performance statistics for distributed GPU optimization

IterationStats

Statistics for individual iterations

Enums

GpuCommunicationStrategy

GPU communication strategies for distributed systems