Crate guard_trait

This crate provides a guarding mechanism for memory, with an interface that is in some ways similar to core::pin::Pin.

Motivation

What this crate attempts to solve, is the problem that data races can occur for memory that is shared with another process or the kernel (via io_uring for instance). If the memory is still shared when the original thread continues to execute after a system call for example, the original buffer can still be accessed while the system call allows the handler to keep using the memory. This does not happen with traditional blocking syscalls; the kernel will only access the memory during the syscall, when the process cannot temporarily do anything else.

However, for more advanced asynchronous interfaces such as io_uring that never copy memory, the memory can still be used once the system call has started, which can lead to data races between two actors sharing memory. To prevent this data race, it is not possible to:

1. Read the memory while it is being written to by the kernel, or write to the memory while it is being read by the kernel. This is exactly like Rust's aliasing rules: we can either allow both the kernel and this process to read a buffer, for example in the system call write(2), we can temporarily give the kernel exclusive ownership one or more buffers when the kernel is going to write to them, or we can avoid sharing memory at all with the kernel, but we cannot let either actor have mutable access while the other has any access at all. (aliasing invariant)
2. Reclaim the memory while it is being read from or written to by the kernel. This is as simple as it sounds: we simply do not want the buffers to be used for other purposes, either by returning the memory to the heap, where it can be allocated simply so that the kernel can overwrite it when it is not supposed to, or it can corrupt stack variables. (reclamation invariant)

The term "kernel" does not necessarily have to be the other actor that the memory is shared with; on Redox for example, the io_uring interface can work solely between regular userspace processes. Additionally, although being a somewhat niche case, this can also be used for safe wrappers protecting memory for DMA in device drivers, with a few additional restrictions (regarding cache coherency) to make that work.

This buffer sharing logic does unfortunately not play very well with the current asynchronous ecosystem, where almost all I/O is done using regular borrowed slices, and references are merely borrows which are cancellable at any time, even by leaking. This functions perfectly when you use synchronous (but non-blocking) system calls where either the process or the kernel can execute at a time. In contrast, io_uring is asynchronous, meaning that the kernel can read and write to buffers, while our program is executing. Therefore, a future that locally stores an array, aliased by the kernel in io_uring, cannot stop the kernel from using the memory again in any reasonable way, if the future were to be Dropped, without blocking indefinitely. What is even worse, is that futures can be leaked at any time, and arrays allocated on the stack can also be dropped, when the memory is still in use by the kernel, as a buffer to write data from e.g. a socket. If a (mutable) buffer on the stack is then reused later for regular variables... arbitrary program corruption!

What we need in order to solve these two complications, is some way to be able to mark a memory region as both "borrowed by the kernel" (mutably or immutably), and "undroppable". Since the Rust borrow checker is smart, any mutable reference with a lifetime that is shorter than 'static, can trivially be leaked, and the pointer can be used again. This rules out any reference of lifetime 'a that 'static outlives, as those may be used again outside of the borrow, potentially mutably. Immutable static references are however completely harmless, since they cannot be dropped nor accessed mutably, and immutable aliasing is always permitted.

Consequently, all buffers that are going to be used in safe code, must be owned. This either means heap-allocated objects (since we can assume that the heap as a whole has the 'static lifetime, and allocations stay forever, until deallocated explicitly), buffer pools which themselves have a guarding mechanism, and static references (both mutable and immutable). We can however allow borrowed data as well, but because of the semantics around lifetimes, and the very fact that the compiler has no idea that the kernel is also involved, that requires unsafe code.

Consider reading "Mental experiments with io_uring", and "Notes on io-uring" for more information about these challenges.

Interface

The way guard_trait solves this, is by adding two simple traits: Guarded and GuardedMut. Guarded is automatically implemented for every pointer type that implements Deref, StableDeref and 'static. Similarly, GuardedMut is implemented under the same conditions, and provided that the pointer implements DerefMut. A consequence of this, is that nearly all owned container types, such as Arc, Box, Vec, etc., all implement the traits, and can thus be used with completion-based interfaces.

For scenarios where it is impossible to ensure at the type level, that a certain pointer follows the guard invariants, AssertSafe also exists, but is unsafe to initialize.

Buffers can also be mapped in a self-referencial way, similar to how owning-ref works, using GuardedExt::map and GuardedMutExt::map_mut. This is especially important when slice indexing is needed, as the only way to limit the number of bytes to do I/O with, generally is to shorten the slice.

Re-exports

pub extern crate stable_deref_trait;

Structs

AssertSafe	A type for pointers that cannot uphold the necessary guard invariants at the type level, but which can be assumed to behave properly by unsafe code.
Mapped	A mapped guard, which contains a guarded owned pointer, and an immutable reference to that pointer. It has had a one-time closure applied to it, but only the output of the closure is stored, not the closure itself. This is similar to how crates like owning_ref work.
MappedMut	A mapped guard, which contains a guarded owned pointer, and a mutable reference to that pointer. It has had a one-time closure applied to it, but only the output of the closure is stored, not the closure itself. This is similar to how crates like owning_ref work.

Traits

Guarded	A trait for pointer types that uphold the guard invariants, namely that the pointer must be owned, and that it must dereference into a stable address.
GuardedExt	An extension trait for convenience methods, that is automatically implemented for all Guarded types.
GuardedMut	A trait for pointer types that uphold the guard invariants, and are able to dereference mutably.
GuardedMutExt	An extension trait for convenience methods, that is automatically implemented for all GuardedMut types.
StableDeref	An unsafe marker trait for types that deref to a stable address, even when moved. For example, this is implemented by Box, Vec, Rc, Arc and String, among others. Even when a Box is moved, the underlying storage remains at a fixed location.