Struct ExternalDeviceMemory 

pub struct ExternalDeviceMemory { /* private fields */ }

Wrapper for externally-allocated device (GPU) memory.

This type wraps a raw pointer to GPU memory that is owned by an external framework (like vLLM). It provides the necessary traits for NIXL registration without taking ownership of the underlying memory.

Safety

This type relies on the caller to guarantee that:

• The pointer points to valid GPU memory on the specified device
• The memory remains valid for the lifetime of this wrapper
• The memory size is exactly as specified
• The external framework doesn’t free the memory while this wrapper exists

Example

// vLLM allocates KV cache tensors
let tensor_ptr = tensor.data_ptr();
let tensor_size = tensor.size_bytes();
let device_id = tensor.device.index;

// Wrap without taking ownership
let external = unsafe {
    ExternalDeviceMemory::new(tensor_ptr as *const u8, tensor_size, device_id as u64)
};

// Register with NIXL for RDMA
let registered = register_with_nixl(external, &agent, None)?;

Implementations

impl ExternalDeviceMemory

pub unsafe fn new(ptr: *const u8, size: usize, device_id: u64) -> Self

Create a wrapper for external device memory.

Safety

Caller must ensure:

• ptr points to valid GPU memory on CUDA device device_id
• The memory remains valid for the lifetime of this wrapper
• The memory size is exactly size bytes
• The external framework doesn’t free the memory while this wrapper exists

pub fn as_ptr(&self) -> *const u8

Get the raw pointer to the external memory.

pub fn device_id(&self) -> u64

Get the CUDA device ID where this memory resides.

Trait Implementations

impl Debug for ExternalDeviceMemory

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl MemoryDescriptor for ExternalDeviceMemory

fn addr(&self) -> usize

Base address of the memory region.

fn size(&self) -> usize

Size of the memory region in bytes.

fn storage_kind(&self) -> StorageKind

Type of storage backing this region.

fn as_any(&self) -> &dyn Any

Enable downcasting to concrete type.

fn nixl_descriptor(&self) -> Option<NixlDescriptor>

Get the NIXL descriptor for this memory region.

impl NixlCompatible for ExternalDeviceMemory

fn nixl_params(&self) -> (*const u8, usize, MemType, u64)

Get parameters needed for NIXL registration.

impl Send for ExternalDeviceMemory

impl Sync for ExternalDeviceMemory