Crate async_rdma

Expand description

RDMA high-level abstraction, providing several useful APIs.

Async-rdma is a framework for writing asynchronous rdma applications with the Rust programing language. At a high level, it provides a few major components:

Tools for establishing connections with rdma endpoints such as RdmaBuilder.
High-level APIs for data transmission between endpoints including read, write, send, receive.
High-level APIs for rdma memory region management including alloc_local_mr, request_remote_mr, send_mr, receive_local_mr, receive_remote_mr.
A framework including agent and event_listener working behind APIs for memory region management and executing rdma requests such as post_send and poll.

Example

A simple example: client request a remote memory region and put data into this remote memory region by rdma write. And finally client send_mr to make server aware of this memory region. Server receive_local_mr, and then get data from this mr.

use async_rdma::{LocalMrReadAccess, LocalMrWriteAccess, Rdma, RdmaListener};
use portpicker::pick_unused_port;
use std::{
    alloc::Layout,
    io,
    net::{Ipv4Addr, SocketAddrV4},
    time::Duration,
};

struct Data(String);

async fn client(addr: SocketAddrV4) -> io::Result<()> {
    let rdma = Rdma::connect(addr, 1, 1, 512).await?;
    let mut lmr = rdma.alloc_local_mr(Layout::new::<Data>())?;
    let mut rmr = rdma.request_remote_mr(Layout::new::<Data>()).await?;
    // load data into lmr
    unsafe { *(*lmr.as_mut_ptr() as *mut Data) = Data("hello world".to_string()) };
    // write the content of local mr into remote mr
    rdma.write(&lmr, &mut rmr).await?;
    // then send rmr's metadata to server to make server aware of it
    rdma.send_remote_mr(rmr).await?;
    Ok(())
}

#[tokio::main]
async fn server(addr: SocketAddrV4) -> io::Result<()> {
    let rdma_listener = RdmaListener::bind(addr).await?;
    let rdma = rdma_listener.accept(1, 1, 512).await?;
    // receive the metadata of the mr sent by client
    let lmr = rdma.receive_local_mr().await?;
    // print the content of lmr, which was `write` by client
    unsafe { println!("{}", &*(*(*lmr.as_ptr() as *const Data)).0) };
    // wait for the agent thread to send all reponses to the remote.
    tokio::time::sleep(Duration::from_secs(1)).await;
    Ok(())
}
#[tokio::main]
async fn main() {
    let addr = SocketAddrV4::new(Ipv4Addr::new(127, 0, 0, 1), pick_unused_port().unwrap());
    std::thread::spawn(move || server(addr));
    tokio::time::sleep(Duration::from_secs(3)).await;
    client(addr)
        .await
        .map_err(|err| println!("{}", err))
        .unwrap();
}