pub struct MemoryPool { /* private fields */ }Implementations§
Source§impl MemoryPool
impl MemoryPool
Sourcepub fn create(props: MemoryPoolProps) -> Result<Self>
pub fn create(props: MemoryPoolProps) -> Result<Self>
Creates a CUDA memory pool.
props determines the properties of the pool such as the backing device and IPC capabilities.
To create a memory pool for host memory not targeting a specific NUMA node, applications must set MemoryPoolProps::location to MemoryLocationKind::Host.
MemoryLocation::id is ignored for such pools.
Pools created with MemoryLocationKind::Host are not IPC-capable and MemoryPoolProps::handle_type must be MemoryAllocationHandleType::None; any other value returns crate::error::Status::InvalidValue.
To create a memory pool targeting a specific host NUMA node, applications must set MemoryLocation::kind to MemoryLocationKind::Numa and MemoryLocation::id must specify the NUMA ID of the host memory node.
Specifying MemoryLocationKind::NumaCurrent as MemoryLocation::kind returns crate::error::Status::InvalidValue.
By default, the pool’s memory is accessible from the device where it is allocated.
Pools created with MemoryLocationKind::Numa or MemoryLocationKind::Host are accessible from the host CPU by default.
Applications can control the maximum size of the pool by specifying a non-zero value for MemoryPoolProps::max_size.
A value of 0 uses a system-dependent maximum pool size.
Callers that intend to use MemoryAllocationHandleType::Fabric based memory sharing must ensure: (1) the nvidia-caps-imex-channels character device is created by the driver and is listed under /proc/devices; (2) at least one IMEX channel file is accessible to the process.
When exporter and importer CUDA processes have been granted access to the same IMEX channel, they can securely share memory.
The IMEX channel security model works per operating-system account. All processes for an account can share memory if that account has access to a valid IMEX channel. When isolation between accounts is desired, each account needs a separate IMEX channel.
These channel files exist in /dev/nvidia-caps-imex-channels/channel* and can be created using standard OS native calls like mknod on Linux.
To create a managed memory pool, applications must set MemoryPoolProps::alloc_type to MemoryAllocationType::Managed.
MemoryPoolProps::handle_type must also be MemoryAllocationHandleType::None because IPC is not supported.
For managed memory pools, MemoryPoolProps::location is treated as the preferred location for all allocations created from the pool.
An application can also set MemoryLocationKind::Invalid to indicate no preferred location.
MemoryPoolProps::max_size must be set to zero for managed memory pools.
MemoryPoolProps::usage must be zero because decompression for managed memory is not supported.
For managed memory pools, all devices on the system must have non-zero concurrentManagedAccess.
If not, this call returns crate::error::Status::NotSupported.
Specifying MemoryAllocationHandleType::None creates a memory pool that does not support IPC.
§Errors
Returns an error if props describes an unsupported pool, CUDA cannot
create the pool, or CUDA returns a null memory-pool handle.
Sourcepub fn set_attribute(
&mut self,
attribute: MemoryPoolAttribute,
value: MemoryPoolAttributeValue,
) -> Result<()>
pub fn set_attribute( &mut self, attribute: MemoryPoolAttribute, value: MemoryPoolAttributeValue, ) -> Result<()>
Supported attributes are:
MemoryPoolAttribute::ReleaseThreshold: amount of reserved memory, in bytes, to keep before trying to release memory back to the OS. When more than the release threshold bytes of memory are held by the memory pool, the allocator will try to release memory back to the OS on the next call to stream, event or context synchronize. (default 0)MemoryPoolAttribute::ReuseFollowEventDependencies: allowssys::cuMemAllocAsyncto use memory asynchronously freed in another stream as long as a stream ordering dependency of the allocating stream on the free action exists. CUDA events and null stream interactions can create the required stream ordered dependencies. (default enabled)MemoryPoolAttribute::ReuseAllowOpportunistic: allows reuse of already completed frees when there is no dependency between the free and allocation. (default enabled)MemoryPoolAttribute::ReuseAllowInternalDependencies: allowssys::cuMemAllocAsyncto insert new stream dependencies to establish the stream ordering required to reuse a piece of memory released bysys::cuMemFreeAsync(default enabled).MemoryPoolAttribute::ReservedMemoryHigh: resets the high watermark that tracks the amount of backing memory allocated for the memory pool. It is illegal to set this attribute to a non-zero value.MemoryPoolAttribute::UsedMemoryHigh: resets the high watermark that tracks the amount of used memory allocated for the memory pool.
Sourcepub fn attribute(
&self,
attribute: MemoryPoolAttribute,
) -> Result<MemoryPoolAttributeValue>
pub fn attribute( &self, attribute: MemoryPoolAttribute, ) -> Result<MemoryPoolAttributeValue>
Supported attributes are:
MemoryPoolAttribute::ReleaseThreshold: amount of reserved memory, in bytes, to keep before trying to release memory back to the OS. When more than the release threshold bytes of memory are held by the memory pool, the allocator will try to release memory back to the OS on the next call to stream, event or context synchronize. (default 0)MemoryPoolAttribute::ReuseFollowEventDependencies: allowssys::cuMemAllocAsyncto use memory asynchronously freed in another stream as long as a stream ordering dependency of the allocating stream on the free action exists. CUDA events and null stream interactions can create the required stream ordered dependencies. (default enabled)MemoryPoolAttribute::ReuseAllowOpportunistic: allows reuse of already completed frees when there is no dependency between the free and allocation. (default enabled)MemoryPoolAttribute::ReuseAllowInternalDependencies: allowssys::cuMemAllocAsyncto insert new stream dependencies to establish the stream ordering required to reuse a piece of memory released bysys::cuMemFreeAsync(default enabled).MemoryPoolAttribute::ReservedMemoryCurrent: backing memory currently allocated for the memory pool.MemoryPoolAttribute::ReservedMemoryHigh: high watermark of backing memory allocated for the memory pool since the last reset.MemoryPoolAttribute::UsedMemoryCurrent: memory from the pool that is currently in use by the application.MemoryPoolAttribute::UsedMemoryHigh: high watermark of memory from the pool that was in use by the application.
§Errors
Returns an error if CUDA Driver cannot report the requested pool attribute.
Sourcepub fn set_access(
&mut self,
access_descs: &[MemoryAccessDescriptor],
) -> Result<()>
pub fn set_access( &mut self, access_descs: &[MemoryAccessDescriptor], ) -> Result<()>
Controls visibility of pools between devices.
§Errors
Returns an error if CUDA Driver rejects the access descriptors.
Sourcepub fn access(&self, location: MemoryLocation) -> Result<MemoryAccessFlag>
pub fn access(&self, location: MemoryLocation) -> Result<MemoryAccessFlag>
Returns the accessibility of the pool’s memory from the specified location.
§Errors
Returns an error if CUDA Driver cannot report access from location.
Sourcepub fn trim_to(&mut self, min_bytes_to_keep: usize) -> Result<()>
pub fn trim_to(&mut self, min_bytes_to_keep: usize) -> Result<()>
Releases memory back to the OS until the pool contains fewer than min_bytes_to_keep reserved bytes, or there is no more memory that the allocator can safely release.
The allocator cannot release OS allocations that back outstanding asynchronous allocations.
The OS allocations may happen at different granularity from the caller’s allocations.
- Allocations that have not been freed count as outstanding.
- Allocations that have been asynchronously freed but whose completion has not been observed on the host, for example by synchronization, can count as outstanding.
§Errors
Returns an error if CUDA cannot trim the pool.