Crate guard_trait

This crate provides a guarding mechanism for memory, with an interface that is in some ways similar to core::pin::Pin.

Motivation

What this crate attempts to solve, is the problem that data races can occur for memory that is shared with another process or the kernel (via io_uring for instance). If the memory is still shared when the original thread continues to execute after a system call for example, the original buffer can still be accessed while the system call allows the handler to keep using the memory. This does not happen with traditional blocking syscalls; the kernel will only access the memory during the syscall, when the process cannot temporarily do anything else.

However, for more advanced asynchronous interfaces such as io_uring that never copy memory, the memory can still be used once the system call has started, which can lead to data races between two actors sharing memory. To prevent this data race, it is not possible to:

1. Read the memory while it is being written to by the kernel, or write to the memory while it is being read by the kernel. This is exactly like Rust's aliasing rules: we can either allow both the kernel and this process to read a buffer, for example in the system call write(2), we can temporarily give the kernel exclusive ownership one or more buffers when the kernel is going to write to them, or we can avoid sharing memory at all with the kernel, but we cannot let either actor have mutable access while the other has any access at all. (aliasing invariant)
2. Reclaim the memory while it is being read from or written to by the kernel. This is as simple as it sounds: we simply do not want the buffers to be used for other purposes, either by returning the memory to the heap, where it can be allocated simply so that the kernel can overwrite it when it is not supposed to, or it can corrupt stack variables. (reclamation invariant)

The term "kernel" does not necessarily have to be the other actor that the memory is shared with; on Redox for example, the io_uring interface can work solely between regular userspace processes. Additionally, although being a somewhat niche case, this can also be used for safe wrappers protecting memory for DMA in device drivers, with a few additional restrictions (regarding cache coherency) to make that work.

This buffer sharing logic does unfortunately not play very well with the current asynchronous ecosystem, where almost all I/O is done using regular borrowed slices, and references are merely borrows which are cancellable at any time, even by leaking. This functions perfectly when you use synchronous (but non-blocking) system calls where either the process or the kernel can execute at a time. In contrast, io_uring is asynchronous, meaning that the kernel can read and write to buffers, while our program is executing. Therefore, a future that locally stores an array, aliased by the kernel in io_uring, cannot stop the kernel from using the memory again in any reasonable way, if the future were to be Dropped, without blocking indefinitely. What is even worse, is that futures can be leaked at any time, and arrays allocated on the stack can also be dropped, when the memory is still in use by the kernel, as a buffer to write data from e.g. a socket. If a (mutable) buffer of a stack is then used for regular variables... arbitrary program corruption!

What we need in order to solve these two complications, is some way to be able to mark a memory region as both "borrowed by the kernel" (mutably or immutably), and "undroppable". Since the Rust borrow checker is smart, any mutable reference with a lifetime that is shorter than 'static, can trivially be leaked, and the pointer can be used again. This rules out any reference of lifetime 'a that 'static outlives, as those may be used again outside of the borrow, potentially mutably. Immutable static references are however completely harmless, since they cannot be dropped nor accessed mutably, and immutable aliasing is always permitted.

Consequently, all buffers that are going to be used in safe code, must be owned. This either means heap-allocated objects (since we can assume that the heap as a whole has the 'static lifetime, and allocations stay forever, until deallocated explicitly), buffer pools which themselves have a guarding mechanism, and static references (both mutable and immutable). We can however allow borrowed data as well, but because of the semantics around lifetimes, and the very fact that the compiler has no idea that the kernel is also involved, that requires unsafe code.

Consider reading "Mental experiments with io_uring", and "Notes on io-uring" for more information about these challenges.

Interface

The main type that this crate provides, is the Guarded struct. Guarded is a wrapper that encapsulates any stable pointer (which in safe code, can only be done for types that implement StableDeref). The reason for StableDeref is because the memory must point to the same buffer, and be independent of the location of the pointer on the stack. Due to this, a newtype that simply borrows an inner field cannot safely be stored in the Guarded wrapper, but heap containers such as std::vec::Vec and std::boxed::Box implement that trait.

Once the wrapper is populated with an active guard, which typically happens when an io_uring opcode is submitted, that will access memory, the wrapper will neither allow merely accessing the memory, nor dropping it, until the guard successfully returns true after calling Guard::try_release. It is then the responsibility of the guard, to make sure dynamically that the memory is no longer aliased. There exist a fallible reclamation method, and a method for trying to access the memory as well, both of which will succeed if the guard is either nonpresent, or if Guard::try_release succeeds. The Drop impl will leak the memory entirely (by not calling the Drop handler of the inner value), if the guard was not able to release the memory when the buffer goes out of scope. It is thus highly advised to manually keep track of the buffer, to prevent accidental leakage.

There are a few corner-cases to these quite strict rules: while they are required for upholding the reclamation invariant, the no-access restriction is not necessary for usages that only read memory. Consider a write(2) system call for instance, which will never write to the buffer, allowing the memory to be accessed immutably while the guard is active, or mutably once the guard is released. Because of this, there are marker types marker::Shared and marker::Exclusive, as generic parameters to Guarded. If marker::Shared is used, the memory can always be accessed read-only.

Static references are special in the way that they are guaranteed to never be dropped (at least in Safe Rust), and they obviously include things like string literals and other static data. However, a guard is still necessary for mutable references, since they could be written to when they should not, if they were to be moved out from the wrapper.

Furthermore, there are some types that handle their guarding mechanism themselves, unlike the Guarded wrapper. The regular Guarded wrapper assumes that all data comes from the heap, or some other global location that persists for the duration of the program, but is suboptimal for custom allocators e.g. in buffer pools. To address this limitation, the Guardable trait exists to abstracts the role of what the Guarded wrapper does, namely being able to insert a guard, and then protect the memory until the guard can be released. For flexibilility, prefer impl Guardable rather than Guarded, if possible. This trait is implemented by Guarded, and the most notable example outside of this crate is BufferSlice from redox-buffer-pool, that needs to tell the buffer pool (which itself is dynamically allocated) that the pool has guarded memory, to prevent it from deallocating that that particular slice, or the pool as a whole.

This interface is conceptually roughly analoguous to the std::pin::Pin API offered by core and std; there is a wrapper Guarded that encapsulates an pointer, and a trait for types that can safely be released from the guard. However, there are is also a major difference: Pin types do not prevent memory reclamation on the stack by leaking, nor mutable aliasing, whatsoever, making them unusable for io_uring. The only thing they ensure, is that the inner type has to have a stable address (i.e. no moving out) before dropping. Meanwhile, Guarded will enforce that one can share an address with another process or hardware, that has no knowledge of when the buffer is getting reclaimed.

Re-exports

pub extern crate stable_deref_trait;

Modules

marker

A module for markers, mainly intended to distinguish between shared (read-only) and exclusive (write capable) memory.

Structs

Guarded	A wrapper for types that can be "guarded", meaning that the memory they point to cannot be safely reclaimed or even moved out, until the guard frees it. For exclusive-access buffers (which are written to, in other words), the data is also completely unaccessible until the guard is released.
TryUnguardError	The error returned from Guarded::try_unguard if the guard could not release the memory region.

Enums

NoGuard

A no-op guard, that cannot be initialized but still useful in type contexts. This the recommended placeholder for types that do not need guarding.

Traits

Guard	A trait for guards, that decide whether memory can be deallocated, or whether it may be shared with an actor outside of control from this process at all.
Guardable	A trait for types that can be "guardable", meaning that they only leak on Drop unless they can remove their guard, that their memory cannot be read from if the kernel may mutate it, and that the memory cannot be written to when the kernel may read it.
StableDeref	An unsafe marker trait for types that deref to a stable address, even when moved. For example, this is implemented by Box, Vec, Rc, Arc and String, among others. Even when a Box is moved, the underlying storage remains at a fixed location.