Skip to main content

Module numa_buffer

Module numa_buffer 

Source
Expand description

NUMA-aware host allocation bookkeeping.

On multi-socket hosts, page-locked memory used for host↔device DMA performs best when it is physically resident on the NUMA node electrically closest to the target GPU (i.e. the node whose PCIe root complex hosts the device). libnuma’s numa_alloc_onnode lets an application bind an allocation to a specific node; the matching CUDA pattern is to pin that allocation with cuMemHostRegister.

This module models the host-side topology and policy required to make that decision without a GPU present: a NumaTopology describing nodes and the NUMA distance matrix, a closest_node_to_gpu selector, and a NumaBuffer descriptor that records which node an allocation is bound to together with its byte footprint. The physical numa_alloc_onnode / cuMemHostRegister calls require the real platform and live behind the device-gated path; everything here is deterministic and unit-testable.

NUMA distances follow the ACPI SLIT convention: the distance from a node to itself is 10, and remote distances are larger multiples (a one-hop remote node is typically 20, i.e. ~2× local latency).

§Example

// Two-socket box: node 0 and node 1, remote distance 20.
let topo = NumaTopology::two_node(20);
// A GPU attached to node 1's root complex should bind host memory to node 1.
let node = closest_node_to_gpu(&topo, 1).expect("node");
assert_eq!(node, 1);

let buf = NumaBuffer::on_node(&topo, node, 4096).expect("alloc");
assert_eq!(buf.node(), 1);
assert_eq!(buf.byte_size(), 4096);

Structs§

NumaAllocTracker
Per-node accounting of NUMA-bound host allocations.
NumaBuffer
Host-side descriptor for a host allocation bound to a NUMA node.
NumaTopology
A model of the host NUMA topology.

Constants§

LOCAL_NUMA_DISTANCE
The ACPI SLIT local-node distance (node to itself).

Functions§

closest_node_to_gpu
Returns the NUMA node closest to a GPU whose PCIe root complex is attached to gpu_home_node.