Crate completeio

CompleteIo

A thread-per-core Rust IO drivers and async runtime backed by IOCP/io_uring/mio. The name comes from “completion-based IO”.

This repository is a fork of compio.

Why not Compio?

The project has different goals:

• provide low overhead IO drivers that preallocate memory on initialization

• drivers API accepts non-’static IO buffers

  • drivers don’t own buffers
  • buffers are required to be Unpin

• bias towards IoUring API to achieve zero-cost abstraction on Linux:

  • fixed size submission queue for operations
  • external runtime could submit an external queue of not yet queued operations as a single batch
  • timers are exposed as Timeout operation and use suspend-aware CLOCK_BOOTTIME clock source when it’s available

• Async runtime is an example runtime to test implementation of drivers

Quick start

With runtime feature enabled, we can use the high level APIs to perform fs & net IO.

use completeio::{fs::File, task::block_on};

let buffer = block_on(async {
    let file = File::open("Cargo.toml").unwrap();
    let (read, buffer) = file.read_to_end_at(Vec::with_capacity(1024), 0).await;
    let read = read.unwrap();
    assert_eq!(read, buffer.len());
    String::from_utf8(buffer).unwrap()
});
println!("{}", buffer);

While you can also control the low-level driver manually:

use arrayvec::ArrayVec;
use std::collections::VecDeque;
use completeio::{
    buf::IntoInner,
    driver::{AsRawFd, Driver, Entry, CompleteIo},
    fs::File,
    op::ReadAt,
};

let mut driver = Driver::new().unwrap();
let file = File::open("Cargo.toml").unwrap();
// Attach the `RawFd` to driver first.
driver.attach(file.as_raw_fd()).unwrap();

// Create operation and push it to the driver.
let mut op = ReadAt::new(file.as_raw_fd(), 0, Vec::with_capacity(4096));
let mut ops = VecDeque::from([(&mut op, 0).into()]);
driver.push_queue(&mut ops);

// Poll the driver and wait for IO completed.
let mut entries = ArrayVec::<Entry, 1>::new();
unsafe {
    driver
        .submit_and_wait_completed(None, &mut entries)
        .unwrap();
}
let entry = entries.drain(..).next().unwrap();
assert_eq!(entry.user_data(), 0);

// Resize the buffer by return value.
let n = entry.into_result().unwrap();
let mut buffer = op.into_inner();
unsafe {
    buffer.set_len(n);
}

println!("{}", String::from_utf8(buffer).unwrap());

Modules

buf

Utilities for working with buffers.

driver

The platform-specified driver. Some types differ by compilation target.

event

Asynchronous events.

fs

Filesystem manipulation operations.

net

Network related.

op

The async operations. Types in this mod represents the low-level operations passed to kernel. The operation itself doesn’t perform anything. You need to pass them to crate::driver::Driver, and poll the driver.

signal

Asynchronous signal handling.

task

The runtime of completeio. We don’t expose the runtime struct because there could be only one runtime in each thread.

time

Utilities for tracking time.

Type Aliases

BufResult

A specialized Result type for operations with buffers.