Module numa 

NUMA-aware memory allocation utilities.

This module provides utilities for NUMA-aware memory allocation, which is critical for optimal performance on multi-socket systems with GPUs. Memory allocated on the NUMA node closest to the target GPU has significantly lower access latency.

Architecture

• NumaNode: Represents a NUMA node ID
• topology: Reads CPU-to-NUMA mapping from /sys/devices/system/node
• worker_pool: Dedicated worker threads pinned to specific NUMA nodes

Usage

NUMA optimization is enabled by default. To disable it:

export DYN_MEMORY_DISABLE_NUMA=1

When enabled, pinned memory allocations are routed through NUMA workers that are pinned to the target GPU’s NUMA node, ensuring first-touch policy places pages on the correct node. If the GPU’s NUMA node cannot be determined, allocation falls back to the non-NUMA path transparently.

Modules

topology

NUMA topology detection

worker_pool

NUMA worker pool for memory allocation with first-touch policy.

Structs

NumaNode

Represents a NUMA node identifier.

Functions

get_current_cpu_numa_node

Get the current CPU’s NUMA node.

get_device_cpu_set

Get a deterministic CPU subset for a CUDA device, subdivided among ALL GPUs sharing the same NUMA node (including those hidden by CUDA_VISIBLE_DEVICES).

get_device_numa_node

Get NUMA node for a GPU device.

is_numa_disabled

Convenience inverse of is_numa_enabled.

is_numa_enabled

Check if NUMA optimization is disabled via environment variable.

pin_thread_to_numa_node

Pin the current thread to a specific NUMA node’s CPUs.