Crate async_rdma

RDMA high-level abstraction, providing several useful APIs.

Async-rdma is a framework for writing asynchronous rdma applications with the Rust programing language. At a high level, it provides a few major components:

• Tools for establishing connections with rdma endpoints such as RdmaBuilder.

• High-level APIs for data transmission between endpoints including read, write, send, receive.

• High-level APIs for rdma memory region management including alloc_local_mr, request_remote_mr, send_mr, receive_local_mr, receive_remote_mr.

• A framework including agent and event_listener working behind APIs for memory region management and executing rdma requests such as post_send and poll.

Example

A simple example: client request a remote memory region and put data into this remote memory region by rdma write. And finally client send_mr to make server aware of this memory region. Server receive_local_mr, and then get data from this mr.

use async_rdma::{Rdma, RdmaListener};
use std::{alloc::Layout, sync::Arc, io, time::Duration, net::{Ipv4Addr, SocketAddrV4}};
use portpicker::pick_unused_port;

struct Data(String);

async fn client(addr: SocketAddrV4) -> io::Result<()> {
    let rdma = Rdma::connect(addr, 1, 1, 512).await?;
    let mut lmr = rdma.alloc_local_mr(Layout::new::<Data>())?;
    let rmr = Arc::new(rdma.request_remote_mr(Layout::new::<Data>()).await?);
    // then send this mr to server to make server aware of this mr.
    unsafe { *(lmr.as_mut_ptr() as *mut Data) = Data("hello world".to_string()) };
    rdma.write(&lmr, &rmr).await?;
    // send the content of lmr to server
    rdma.send_mr(rmr.clone()).await?;
    Ok(())
}

#[tokio::main]
async fn server(addr: SocketAddrV4) -> io::Result<()> {
    let rdma_listener = RdmaListener::bind(addr).await?;
    let rdma = rdma_listener.accept(1, 1, 512).await?;
    let lmr = rdma.receive_local_mr().await?;
    // print the content of lmr, which was `write` by client
    unsafe { println!("{}", &*(*(lmr.as_ptr() as *const Data)).0) };
    Ok(())
}
#[tokio::main]
async fn main() {
    let addr = SocketAddrV4::new(Ipv4Addr::new(127, 0, 0, 1), pick_unused_port().unwrap());
    std::thread::spawn(move || server(addr));
    tokio::time::sleep(Duration::new(1, 0)).await;
    client(addr).await.map_err(|err| println!("{}", err)).unwrap();
}

Structs

Rdma

Rdma handler, the only interface that the users deal with rdma

RdmaBuilder

The builder for the Rdma, it follows the builder pattern.

RdmaListener

Rdma Listener is the wrapper of a TcpListener, which is used to build the rdma queue pair.

Enums

AccessFlag

A wrapper for ibv_access_flag, hide the ibv binding types

Type Definitions

LocalMemoryRegion

Local Memory Region type

RemoteMemoryRegion

Remote memory region type