Module cratetorrent::iovecs

This crate provides a helper type for a slice of IoVecs for zero-copy functionality to bound iovecs by a byte count and to advance the buffer cursor after partial vectored IO.

Bounding input buffers

This is most useful when writing to a portion of a file while ensuring that the file is not extended if the input buffers are larger than the length of the file slice, which in the torrent scenario may occur if the file is not a multiple of the block size.

If the total size of the buffers exceeds the max length, the buffers are split such that the first half returned is the portion of the buffers that can be written to the file, while the second half is the remainder of the buffer, to be used later. If the size of the total size of the buffers is smaller than or equal to the slice, this is essentially a noop.

In reality, the situation here is more complex than just splitting buffers in half, but this is taken care of by the IoVecs implementation.

However, the abstraction leaks through because the iovec at which the buffers were split may be shrunk to such a size as would enable all buffers to stay within the file slice length. This can be restored using IoVecs::into_tail, but until this is called, the original buffers cannot be used, which is enforced by the borrow checker.

Advancing the write cursor

IO syscalls generally don't guarantee writing or filling the input buffers in one system call. This is why these APIs always return the number of bytes transferred, so that calling code can advance the buffer cursor. This functionality implemented by the IoVecs::advance method, which takes offsets the start of the slices by some number of bytes.

Examples

What follows is a complete example making use of both above mentioned API features.

Visualized, this looks like the following:

------------------------------
| file slice: 25             |
-----------------------------------
| block: 16      | block: 16 ^    |
-----------------------------^-----
                             ^
                         split here

In this example, the first half of the split would be [0, 25), the second half would be [25, 32).

use {
    std::{
        fs::{OpenOptions, File},
        os::unix::io::AsRawFd,
    },
    nix::sys::uio::{pwritev, IoVec},
    cratetorrent::iovecs::IoVecs,
};

let file = OpenOptions::new().write(true).create(true).open("/tmp/example").unwrap();
let file_len = 25;
// total length of buffers is 32
let blocks =
    vec![(0..16).collect::<Vec<u8>>(), (16..32).collect::<Vec<u8>>()];
// the raw data is in vecs but file IO works with iovecs, so we need to make
// at least one allocation here to get a contiguouos memory region of iovecs
let mut bufs: Vec<_> =
    blocks.iter().map(|buf| IoVec::from_slice(&buf)).collect();
let mut iovecs = bufs.as_mut_slice();
// bound iovecs by the file length so that we don't write past its end
let mut write_bufs = IoVecs::bounded(iovecs, file_len);

// get write buffers as an immutable slice of iovecs, which will return the
// first 25 bytes, and write the first 25 bytes of buffers
assert_eq!(write_bufs.as_slice().iter().map(|iov| iov.as_slice().len()).sum::<usize>(), file_len);
let write_count = write_vectored_at(&file, 0, &mut write_bufs);
assert_eq!(write_count, file_len);

// after this is done, we can use the second half of the buffers
iovecs = write_bufs.into_tail();
assert_eq!(iovecs.iter().map(|iov| iov.as_slice().len()).sum::<usize>(), 32 - file_len);

// now we can write the rest of the buffers to another file

/// Writes the slice of iovecs to the file.
fn write_vectored_at<'a>(file: &File, mut offset: i64, iovecs: &mut IoVecs<'a>) -> usize {
    let mut total_write_count = 0;
    while !iovecs.as_slice().is_empty() {
        let write_count = pwritev(
            file.as_raw_fd(),
            iovecs.as_slice(),
            offset,
        )
        .expect("cannot write to file");
        iovecs.advance(write_count);
        total_write_count += write_count;
        offset += write_count as i64;
    }
    total_write_count
}

Structs

IoVec
IoVecs	Wrapper over a slice of IoVecs that provides zero-copy functionality to pass only a subslice of the iovecs to vectored IO functions.

Functions

advance

This function is analogous to IoVecs::advance, except that it works on a list of mutable iovec buffers, while the former is for an immutable list of such buffers.