rdma

Low-level RDMA API.

Environment

This crate requires the latest rdma-core installation.

Please make sure the following commands work:

$ pkg-config --modversion libibverbs
1.14.41.0
$ pkg-config --modversion librdmacm
1.3.41.0

The output also shows the minimum required versions.

Develop

Examples

Add a Soft-RoCE device:

sudo rdma link add rxe0 type rxe netdev ens33

rxe0 is the RDMA device name. You can name it whatever you want. ens33 is the name of a network device. The name of the network device may be different. You can see it by running command ifconfig.

There is a tutorial for Chinese users: https://zhuanlan.zhihu.com/p/361740115.

Please install the tool just if you don't have it.

Install the examples:

just install-examples

Run the example rdma-devices:

$ rdma-devices
|          name          |          guid          |
|------------------------|------------------------|
|          rxe0          |    26418cfffe021df9    |

Run the example rdma-pingpong:

• RC service

  • server side

    rdma-pingpong rc
    

  • client side

    rdma-pingpong rc 127.0.0.1
    

• UD service

  • server side

    rdma-pingpong ud
    

  • client side

    rdma-pingpong ud 127.0.0.1
    

Run the benchmarks:

just bench-pingpong-rc
just bench-pingpong-ud

Memory Management

All the APIs related with raw memory are unsafe.

We are exploring how to provide a safe and easy way to manage memory in RDMA.

Resource Management

The RDMA resources are managed by reference counting.

The graph below shows the relations between resources.