rdmatop

htop, but for RDMA traffic — a real-time TUI monitor for RDMA network interfaces.

Monitors per-device throughput (Gbps, packets/s, drops), RDMA read/write counters, retransmits, health events, and shows which processes are using each RDMA device — all via RDMA netlink, the same interface used by rdma statistic.

Requirements

• Linux (netlink-based — macOS/Windows are not supported)
• RDMA-capable NICs (e.g., Mellanox/NVIDIA ConnectX, AWS EFA)

Installation

Ubuntu (PPA)

On Ubuntu 22.04 (jammy), 24.04 (noble), or 26.04 (resolute):

sudo add-apt-repository ppa:crazyguitar/rdmatop
sudo apt update
sudo apt install rdmatop

Cargo

cargo install rdmatop

From source

make         # cargo build
make install # cargo install

Usage

rdmatop

Examples

Use rdmatop to monitor RDMA traffic while running GPU communication benchmarks:

• IB Perftest — two-node ib_write_bw benchmark
• UCX Perftest — two-node ucx_perftest bandwidth / latency
• NCCL — collective communication
• NIXL — point-to-point KV cache transfer
• NVSHMEM — one-sided GPU communication
• PPLX Kernels — MoE all-to-all dispatch/combine
• UCCL — DeepEP-compatible expert-parallel dispatch/combine
• RDMA Statistics — shell-based RDMA stats
• Kubernetes — DaemonSet deployment for Kubernetes

How It Works

1. Device enumeration — RDMA_NLDEV_CMD_GET via netlink to discover all RDMA devices
2. HW counters — RDMA_NLDEV_CMD_STAT_GET per device/port, same as rdma statistic show
3. Process detection — RDMA_NLDEV_CMD_RES_QP_GET to map QPs → PIDs, enriched with /proc data
4. Throughput — Two snapshots per interval, delta / elapsed for rates

License

Apache-2.0