Struct DeviceBox

Source

pub struct DeviceBox<T: DeviceCopy> { /* private fields */ }

Expand description

A pointer type for heap-allocation in CUDA device memory.

See the module-level documentation for more information on device memory.

Implementations§

Source §

impl<T: DeviceCopy> DeviceBox<T>

Source

pub fn new(val: &T) -> CudaResult<Self>

Allocate device memory and place val into it.

This doesn’t actually allocate if T is zero-sized.

§Errors

If a CUDA error occurs, return the error.

§Examples

use cust::memory::*;
let five = DeviceBox::new(&5).unwrap();

Source

pub unsafe fn new_async(val: &T, stream: &Stream) -> CudaResult<Self>

Allocates device memory asynchronously and asynchronously copies val into it.

This doesn’t actually allocate if T is zero-sized.

If the memory behind val is not page-locked (pinned), a staging buffer will be allocated using a worker thread. If you are going to be making many asynchronous copies, it is generally a good idea to keep the data as a crate::memory::LockedBuffer or crate::memory::LockedBox. This will ensure the driver does not have to allocate a staging buffer on its own.

However, don’t keep all of your data as page-locked, doing so might slow down the OS because it is unable to page out that memory to disk.

§Safety

This method enqueues two operations on the stream: An async allocation and an async memcpy. Because of this, you must ensure that:

The memory is not used in any way before it is actually allocated on the stream. You can ensure this happens by synchronizing the stream explicitly or using events.
val is still valid when the memory copy actually takes place.

§Examples

use cust::{memory::*, stream::*};
let stream = Stream::new(StreamFlags::DEFAULT, None)?;
let mut host_val = 0;
unsafe {
    let mut allocated = DeviceBox::new_async(&5u8, &stream)?;
    allocated.async_copy_to(&mut host_val, &stream)?;
    allocated.drop_async(&stream)?;
}
// ensure all async ops are done before trying to access the value
stream.synchronize()?;
assert_eq!(host_val, 5);

Source

pub fn drop_async(self, stream: &Stream) -> CudaResult<()>

Enqueues an operation to free the memory backed by this DeviceBox on a particular stream. The stream will free the allocation as soon as it reaches the operation in the stream. You can ensure the memory is freed by synchronizing the stream.

This function uses internal memory pool semantics. Async allocations will reserve memory in the default memory pool in the stream, and async frees will release the memory back to the pool for further use by async allocations.

The memory inside of the pool is all freed back to the OS once the stream is synchronized unless a custom pool is configured to not do so.

§Examples

use cust::{memory::*, stream::*};
let stream = Stream::new(StreamFlags::DEFAULT, None)?;
let mut host_val = 0;
unsafe {
    let mut allocated = DeviceBox::new_async(&5u8, &stream)?;
    allocated.async_copy_to(&mut host_val, &stream)?;
    allocated.drop_async(&stream)?;
}
// ensure all async ops are done before trying to access the value
stream.synchronize()?;
assert_eq!(host_val, 5);

Source §

impl<T: DeviceCopy + Default> DeviceBox<T>

Source

pub fn as_host_value(&self) -> CudaResult<T>

Read the data back from the GPU into host memory.

Source §

impl<T: DeviceCopy + Zeroable> DeviceBox<T>

Source

pub fn zeroed() -> CudaResult<Self>

Available on crate feature bytemuck only.

Allocate device memory and fill it with zeroes (0u8).

This doesn’t actually allocate if T is zero-sized.

§Examples

use cust::memory::*;
let mut zero = DeviceBox::zeroed().unwrap();
let mut value = 5u64;
zero.copy_to(&mut value).unwrap();
assert_eq!(0, value);

Source

pub unsafe fn zeroed_async(stream: &Stream) -> CudaResult<Self>

Available on crate feature bytemuck only.

Allocates device memory asynchronously and asynchronously fills it with zeroes (0u8).

This doesn’t actually allocate if T is zero-sized.

§Safety

This method enqueues two operations on the stream: An async allocation and an async memset. Because of this, you must ensure that:

The memory is not used in any way before it is actually allocated on the stream. You can ensure this happens by synchronizing the stream explicitly or using events.

§Examples

use cust::{memory::*, stream::*};
let stream = Stream::new(StreamFlags::DEFAULT, None)?;
let mut value = 5u64;
unsafe {
    let mut zero = DeviceBox::zeroed_async(&stream)?;
    zero.async_copy_to(&mut value, &stream)?;
    zero.drop_async(&stream)?;
}
stream.synchronize()?;
assert_eq!(value, 0);

Source §

impl<T: DeviceCopy> DeviceBox<T>

Source

pub unsafe fn uninitialized() -> CudaResult<Self>

Allocate device memory, but do not initialize it.

This doesn’t actually allocate if T is zero-sized.

§Safety

Since the backing memory is not initialized, this function is not safe. The caller must ensure that the backing memory is set to a valid value before it is read, else undefined behavior may occur.

§Examples

use cust::memory::*;
let mut five = unsafe { DeviceBox::uninitialized().unwrap() };
five.copy_from(&5u64).unwrap();

Source

pub unsafe fn uninitialized_async(stream: &Stream) -> CudaResult<Self>

Allocates device memory asynchronously on a stream, without initializing it.

This doesn’t actually allocate if T is zero sized.

§Safety

The allocated memory retains all of the unsafety of DeviceBox::uninitialized, with the additional consideration that the memory cannot be used until it is actually allocated on the stream. This means proper stream ordering semantics must be followed, such as only enqueing kernel launches that use the memory AFTER the allocation call.

You can synchronize the stream to ensure the memory allocation operation is complete.

Source

pub unsafe fn from_raw(ptr: CUdeviceptr) -> Self

Constructs a DeviceBox from a raw pointer.

After calling this function, the raw pointer and the memory it points to is owned by the DeviceBox. The DeviceBox destructor will free the allocated memory, but will not call the destructor of T. This function may accept any pointer produced by the cuMemAllocManaged CUDA API call.

§Safety

This function is unsafe because improper use may lead to memory problems. For example, a double free may occur if this function is called twice on the same pointer, or a segfault may occur if the pointer is not one returned by the appropriate API call.

§Examples

use cust::memory::*;
let x = DeviceBox::new(&5).unwrap();
let ptr = DeviceBox::into_device(x).as_raw_mut();
let x = unsafe { DeviceBox::from_raw(ptr) };

Source

pub unsafe fn from_device(ptr: DevicePointer<T>) -> Self

Constructs a DeviceBox from a DevicePointer.

After calling this function, the pointer and the memory it points to is owned by the DeviceBox. The DeviceBox destructor will free the allocated memory, but will not call the destructor of T. This function may accept any pointer produced by the cuMemAllocManaged CUDA API call, such as one taken from DeviceBox::into_device.

§Safety

This function is unsafe because improper use may lead to memory problems. For example, a double free may occur if this function is called twice on the same pointer, or a segfault may occur if the pointer is not one returned by the appropriate API call.

§Examples

use cust::memory::*;
let x = DeviceBox::new(&5).unwrap();
let ptr = DeviceBox::into_device(x);
let x = unsafe { DeviceBox::from_device(ptr) };

Source

pub fn into_device(b: DeviceBox<T>) -> DevicePointer<T>

Consumes the DeviceBox, returning the wrapped DevicePointer.

After calling this function, the caller is responsible for the memory previously managed by the DeviceBox. In particular, the caller should properly destroy T and deallocate the memory. The easiest way to do so is to create a new DeviceBox using the DeviceBox::from_device function.

Note: This is an associated function, which means that you have to all it as DeviceBox::into_device(b) instead of b.into_device() This is so that there is no conflict with a method on the inner type.

§Examples

use cust::memory::*;
let x = DeviceBox::new(&5).unwrap();
let ptr = DeviceBox::into_device(x);

Source

pub fn as_device_ptr(&self) -> DevicePointer<T>

Returns the contained device pointer without consuming the box.

This is useful for passing the box to a kernel launch.

§Examples

use cust::memory::*;
let mut x = DeviceBox::new(&5).unwrap();
let ptr = x.as_device_ptr();
println!("{:p}", ptr);

Source

pub fn drop(dev_box: DeviceBox<T>) -> DropResult<DeviceBox<T>>

Destroy a DeviceBox, returning an error.

Deallocating device memory can return errors from previous asynchronous work. This function destroys the given box and returns the error and the un-destroyed box on failure.

§Example

use cust::memory::*;
let x = DeviceBox::new(&5).unwrap();
match DeviceBox::drop(x) {
    Ok(()) => println!("Successfully destroyed"),
    Err((e, dev_box)) => {
        println!("Failed to destroy box: {:?}", e);
        // Do something with dev_box
    },
}