rdmatop 0.1.23

A real-time TUI monitor for RDMA network interfaces
rdmatop-0.1.23 is not a library.
Visit the last successful build: rdmatop-0.1.32

rdmatop

Crates.io License

htop, but for RDMA traffic — a real-time TUI monitor for RDMA network interfaces.

Monitors per-device throughput (Gbps, packets/s, drops), RDMA read/write counters, retransmits, health events, and shows which processes are using each RDMA device — all via RDMA netlink, the same interface used by rdma statistic.

Requirements

  • Linux (netlink-based — macOS/Windows are not supported)
  • RDMA-capable NICs (e.g., Mellanox/NVIDIA ConnectX, AWS EFA)

Installation

Ubuntu (PPA)

On Ubuntu 22.04 (jammy), 24.04 (noble), or 26.04 (resolute) — amd64 and arm64:

sudo add-apt-repository ppa:crazyguitar/rdmatop
sudo apt update
sudo apt install rdmatop

Cargo

cargo install rdmatop

From source

make         # cargo build
make install # cargo install

Usage

rdmatop

Perfetto recording

Press r in the TUI to start recording and r again to stop. rdmatop captures every device's tx/rx Gbps and packets/s per interval and writes a Chrome-JSON trace (rdmatop-<n>.json) you can drag into ui.perfetto.dev — each device/port becomes its own set of counter tracks. Timestamps are relative to when you pressed r, so the trace spans exactly your record window.

Examples

Use rdmatop to monitor RDMA traffic while running GPU communication benchmarks:

  • PyTorch — intranode NVLink/XGMI traffic
  • IB Perftest — two-node ib_write_bw benchmark
  • UCX Perftest — two-node ucx_perftest bandwidth / latency
  • NCCL — collective communication
  • NIXL — point-to-point KV cache transfer
  • NVSHMEM — one-sided GPU communication
  • PPLX Kernels — MoE all-to-all dispatch/combine
  • UCCL — DeepEP-compatible expert-parallel dispatch/combine
  • RDMA Statistics — shell-based RDMA stats
  • Kubernetes — DaemonSet deployment for Kubernetes

How It Works

  1. Device enumeration — RDMA_NLDEV_CMD_GET via netlink to discover all RDMA devices
  2. HW counters — RDMA_NLDEV_CMD_STAT_GET per device/port, same as rdma statistic show
  3. Process detection — RDMA_NLDEV_CMD_RES_QP_GET to map QPs → PIDs, enriched with /proc data
  4. Throughput — Two snapshots per interval, delta / elapsed for rates

Contributing

See CONTRIBUTING.md for build/test instructions, design ground rules, and how to submit changes.

License

Apache-2.0