pub struct DevicePool { /* private fields */ }Expand description
Per-device context pool for multi-GPU management.
Maintains a mapping from device ordinals to contexts, with thread-safe access for multi-threaded workloads. Each device gets exactly one context, created with default scheduling flags.
§Round-robin scheduling
The next_device method implements
round-robin device selection using an atomic counter, making it safe
to call from multiple threads without locking.
§Best-available scheduling
The best_available_device method
selects the device with the most total memory. In a future release,
this may query free memory at runtime when the driver supports it.
Implementations§
Source§impl DevicePool
impl DevicePool
Sourcepub fn new() -> CudaResult<Self>
pub fn new() -> CudaResult<Self>
Creates a new pool with contexts for all available devices.
Enumerates every CUDA-capable device and creates one context per
device. The contexts are created with default scheduling flags
(crate::context::flags::SCHED_AUTO).
§Errors
CudaError::NoDeviceif no CUDA devices are available.- Other driver errors from device enumeration or context creation.
Sourcepub fn with_devices(devices: &[Device]) -> CudaResult<Self>
pub fn with_devices(devices: &[Device]) -> CudaResult<Self>
Creates a pool with contexts for specific devices.
One context is created per device in the provided slice. The ordering in the slice determines the iteration and round-robin order.
§Errors
CudaError::InvalidValueif the device slice is empty.- Other driver errors from context creation.
Sourcepub fn context(&self, device_ordinal: i32) -> CudaResult<&Arc<Context>>
pub fn context(&self, device_ordinal: i32) -> CudaResult<&Arc<Context>>
Returns the context for the given device ordinal.
Searches the pool for a device whose ordinal matches the given value.
§Errors
Returns CudaError::InvalidDevice if no device with the given
ordinal is in the pool.
Sourcepub fn device_count(&self) -> usize
pub fn device_count(&self) -> usize
Returns the number of devices in the pool.
Sourcepub fn best_available_device(&self) -> CudaResult<Device>
pub fn best_available_device(&self) -> CudaResult<Device>
Returns the device with the most total memory.
This is a heuristic for selecting the “best” device when you want
to maximise available memory. For real-time free-memory queries,
use cuMemGetInfo (once it is wired into the driver API).
§Errors
Returns an error if memory queries fail.
Sourcepub fn next_device(&self) -> CudaResult<Device>
pub fn next_device(&self) -> CudaResult<Device>
Selects a device using round-robin scheduling.
Each call advances an internal atomic counter and returns the next device in sequence. This is safe to call concurrently from multiple threads.
§Errors
This method is infallible for a properly constructed pool, but
returns CudaResult for API consistency.
Sourcepub fn iter(&self) -> impl Iterator<Item = (&Device, &Arc<Context>)>
pub fn iter(&self) -> impl Iterator<Item = (&Device, &Arc<Context>)>
Iterates over all (device, context) pairs in pool order.
Sourcepub fn context_at(&self, index: usize) -> CudaResult<&Arc<Context>>
pub fn context_at(&self, index: usize) -> CudaResult<&Arc<Context>>
Returns the context for the device at the given pool index.
Pool indices are 0-based and correspond to the order in which devices were added to the pool.
§Errors
Returns CudaError::InvalidValue if the index is out of bounds.
Sourcepub fn device_at(&self, index: usize) -> CudaResult<Device>
pub fn device_at(&self, index: usize) -> CudaResult<Device>
Returns the device at the given pool index.
§Errors
Returns CudaError::InvalidValue if the index is out of bounds.