Trait cust::memory::MemoryAdvise

source · [−]
pub trait MemoryAdvise<T: DeviceCopy>: Sealed {
    fn as_slice(&self) -> &[T]ⓘNotable traits for &'_ mut [u8]impl Write for &'_ mut [u8]impl Read for &'_ [u8];

    fn prefetch_to_host(&self, stream: &Stream) -> CudaResult<()> { ... }

    fn prefetch_to_device(
        &self, 
        stream: &Stream, 
        device: &Device
    ) -> CudaResult<()> { ... }

    fn advise_read_mostly(&self, read_mostly: bool) -> CudaResult<()> { ... }

    fn preferred_location(
        &self, 
        preferred_location: Option<Device>
    ) -> CudaResult<()> { ... }

    fn unset_preferred_location(&self) -> CudaResult<()> { ... }
}
Expand description

Functions for advising the driver about certain uses of unified memory. Such as advising the driver to prefetch memory or to treat memory as read-mostly.

Note that none of the following APIs are required for correctness and/or safety, any use of the memory will be valid no matter the use of the following functions. However, such uses may be very inefficient and/or have increased memory consumption.

Required methods

Provided methods

Advises the driver to enqueue an operation on the stream to prefetch the memory to the CPU. This will cause the driver to fetch the data back to the CPU as soon as the operation is reached on the stream.

The CPU must have the attribute DeviceAttribute::ConcurrentManagedAccess.

Example
use cust::memory::*;
let stream = Stream::new(StreamFlags::NON_BLOCKING, None)?;
let x = UnifiedBuffer::from_slice(&[10u32, 20, 30])?;
x.prefetch_to_host(&stream);
stream.synchronize()?;

Advises the driver to enqueue an operation on the stream to prefetch the memory to a certain GPU. This will cause the driver to fetch the data to the specified device as soon as the operation is reached on the stream.

The device must have the attribute DeviceAttribute::ConcurrentManagedAccess.

Example
use cust::memory::*;
let device = Device::get_device(0)?;
let stream = Stream::new(StreamFlags::NON_BLOCKING, None)?;
let x = UnifiedBuffer::from_slice(&[10u32, 20, 30])?;
x.prefetch_to_device(&stream, &device);
stream.synchronize()?;

Advises the driver that this memory range is mostly going to be read to, and occasionally written to.

Any read accesses from any processor will create a read-only copy of at least the accessed pages in that processor’s memory.

Additionally, when prefetching, a read-only copy of the data will be created on the destination processor. If any processor attempts to write to this data, all copies of the corresponding page will be invalidated except for the one where the write occurred.

For a page to be read-duplicated, the accessing processor must have a non-zero value for DeviceAttribute::ConcurrentManagedAccess. Additionally, if a context is created on a device that does not have DeviceAttribute::ConcurrentManagedAccess, then read-duplication will not occur until all such contexts are destroyed.

Advises the driver as to the preferred device for this memory range. Either a device with Some(device) or the CPU with None. If the device is a GPU, it must have DeviceAttribute::ConcurrentManagedAccess.

Setting the preferred location does not cause the data to be migrated to that location immediately. It instead guides the migration policy when a fault occurs on the memory region. If the data is already in its preferred location and the faulting processor can establish a mapping without requiring the data to be migrated, then data migration will be avoided. On the other hand, if the data is not there or a mapping cannot be established, then it will be migrated to the accessing processor.

Having a preferred location can override the page thrash detection and resolution logic in the unified memory driver. Normally if a page is detected to be constantly thrashing between processors, the page may eventually be pinned to host memory by the driver. But if the preferred location is set as device memory, then the page will continue to thrash indefinitely.

If advise_read_mostly is set on this memory region or a subset of it, then the policies associated with that device will override the policies of this advice.

This advice does not prevent the use of prefetch_to_host or prefetch_to_device.

Undoes the most recent changes by preferred_location.

Implementors