Crate async_rdma

Expand description

RDMA high-level abstraction, providing several useful APIs.

Async-rdma is a framework for writing asynchronous rdma applications with the Rust programing language. At a high level, it provides a few major components:

Tools for establishing connections with rdma endpoints such as RdmaBuilder.
High-level APIs for data transmission between endpoints including read, write, send, receive.
High-level APIs for rdma memory region management including alloc_local_mr, request_remote_mr, send_mr, receive_local_mr, receive_remote_mr.
A framework including agent and event_listener working behind APIs for memory region management and executing rdma requests such as post_send and poll.

Example

A simple example: client request a remote memory region and put data into this remote memory region by rdma write. And finally client send_mr to make server aware of this memory region. Server receive_local_mr, and then get data from this mr.

use async_rdma::{LocalMrReadAccess, LocalMrWriteAccess, RdmaBuilder};
use portpicker::pick_unused_port;
use std::{
    alloc::Layout,
    io::{self, Write},
    net::{Ipv4Addr, SocketAddrV4},
    time::Duration,
};

async fn client(addr: SocketAddrV4) -> io::Result<()> {
    let layout = Layout::new::<[u8; 8]>();
    let rdma = RdmaBuilder::default().connect(addr).await?;
    // alloc 8 bytes remote memory
    let mut rmr = rdma.request_remote_mr(layout).await?;
    // alloc 8 bytes local memory
    let mut lmr = rdma.alloc_local_mr(layout)?;
    // write data into lmr
    let _num = lmr.as_mut_slice().write(&[1_u8; 8])?;
    // write the second half of the data in lmr to the rmr
    rdma.write(&lmr.get(4..8).unwrap(), &mut rmr.get_mut(4..8).unwrap())
        .await?;
    // send rmr's meta data to the remote end
    rdma.send_remote_mr(rmr).await?;
    Ok(())
}

#[tokio::main]
async fn server(addr: SocketAddrV4) -> io::Result<()> {
    let rdma = RdmaBuilder::default().listen(addr).await?;
    // receive mr's meta data from client
    let lmr = rdma.receive_local_mr().await?;
    let data = *lmr.as_slice();
    println!("Data written by the client using RDMA WRITE: {:?}", data);
    assert_eq!(data, [[0_u8; 4], [1_u8; 4]].concat());
    Ok(())
}

#[tokio::main]
async fn main() {
    let addr = SocketAddrV4::new(Ipv4Addr::new(127, 0, 0, 1), pick_unused_port().unwrap());
    std::thread::spawn(move || server(addr));
    tokio::time::sleep(Duration::new(1, 0)).await;
    client(addr)
        .await
        .map_err(|err| println!("{}", err))
        .unwrap();
}

Modules

device

The rmda device

Macros

impl_from_buidler_error_for_another

Convert different BuilderErrors. Impl From<XBuilderError> for YBuilderError.

impl_into_io_error

Convert BuilderErrors to io::Error. Impl From<OtherKindError> for io::Error.

impl_sub_builder_setter

Sub builder setter methord helper. Impl From<XBuilderError> for YBuilderError.

Structs

Gid

A 128-bit identifier used to identify a Port on a network adapter, a port on a Router, or a Multicast Group.

LocalMr

Local Memory Region

ManualTrigger

Manual polling trigger.

MrToken

Memory region token used for the remote access

MrTokenBuilder

Builder for MrToken.

QueuePairEndpoint

Queue pair information used to establish connection.

QueuePairEndpointBuilder

Builder for QueuePairEndpoint.

Rdma

Rdma handler, the only interface that the users deal with rdma

RdmaBuilder

The builder for the Rdma, it follows the builder pattern.

RdmaListener

Rdma Listener is the wrapper of a TcpListener, which is used to build the rdma queue pair.

RemoteMr

Remote memory region

Enums

AccessFlag

A wrapper for ibv_access_flag, hide the ibv binding types

ConnectionType

Method of establishing a connection

MRManageStrategy

Strategies to manage MRs

MTU

The path MTU (Maximum Transfer Unit) i.e. the maximum payload size of a packet that can be transferred in the path. For UC and RC QPs, when needed, the RDMA device will automatically fragment the messages to packet of this size.

MrTokenBuilderError

Error type for MrTokenBuilder

PollingTriggerType

Type of polling trigger.

QueuePairState

The state of qp