Struct HostSharedMemory

pub struct HostSharedMemory { /* private fields */ }

A HostSharedMemory allows synchronized accesses to guest communication buffers, allowing it to be used concurrently with a GuestSharedMemory.

Given future requirements for asynchronous I/O with a minimum amount of copying (e.g. WASIp3 streams), we would like it to be possible to safely access these buffers concurrently with the guest, ensuring that (1) data is read appropriately if the guest is well-behaved; and (2) the host’s behaviour is defined regardless of whether or not the guest is well-behaved.

The ideal (future) flow for a guest->host message is something like

• Guest writes (unordered) bytes describing a work item into a buffer
• Guest reveals buffer via a release-store of a pointer into an MMIO ring-buffer
• Host acquire-loads the buffer pointer from the “MMIO” ring buffer
• Host (unordered) reads the bytes from the buffer
• Host performs validation of those bytes and uses them

Unfortunately, there appears to be no way to do this with defined behaviour in present Rust (see e.g. https://github.com/rust-lang/unsafe-code-guidelines/issues/152). Rust does not yet have its own defined memory model, but in the interim, it is widely treated as inheriting the current C/C++ memory models. The most immediate problem is that regardless of anything else, under those memory models [1, p. 17-18; 2, p. 88],

The execution of a program contains a data race if it contains two [C++23: “potentially concurrent”] conflicting actions [C23: “in different threads”], at least one of which is not atomic, and neither happens before the other [C++23: “, except for the special case for signal handlers described below”]. Any such data race results in undefined behavior.

Consequently, if a misbehaving guest fails to correctly synchronize its stores with the host, the host’s innocent loads will trigger undefined behaviour for the entire program, including the host. Note that this also applies if the guest makes an unsynchronized read of a location that the host is writing!

Despite Rust’s de jure inheritance of the C memory model at the present time, the compiler in many cases de facto adheres to LLVM semantics, so it is worthwhile to consider what LLVM does in this case as well. According to the the LangRef [3] memory model, loads which are involved in a race that includes at least one non-atomic access (whether the load or a store) return undef, making them roughly equivalent to reading uninitialized memory. While this is much better, it is still bad.

Considering a different direction, recent C++ papers have seemed to lean towards using volatile for similar use cases. For example, in P1152R0 [4], JF Bastien notes that

We’ve shown that volatile is purposely defined to denote external modifications. This happens for:

• Shared memory with untrusted code, where volatile is the right way to avoid time-of-check time-of-use (ToCToU) races which lead to security bugs such as [PWN2OWN] and [XENXSA155].

Unfortunately, although this paper was adopted for C++20 (and, sadly, mostly un-adopted for C++23, although that does not concern us), the paper did not actually redefine volatile accesses or data races to prevent volatile accesses from racing with other accesses and causing undefined behaviour. P1382R1 [5] would have amended the wording of the data race definition to specifically exclude volatile, but, unfortunately, despite receiving a generally-positive reception at its first WG21 meeting more than five years ago, it has not progressed.

Separately from the data race issue, there is also a concern that according to the various memory models in use, there may be ways in which the guest can semantically obtain uninitialized memory and write it into the shared buffer, which may also result in undefined behaviour on reads. The degree to which this is a concern is unclear, however, since it is unclear to what degree the Rust abstract machine’s conception of uninitialized memory applies to the sandbox. Returning briefly to the LLVM level, rather than the Rust level, this, combined with the fact that racing loads in LLVM return undef, as discussed above, we would ideally llvm.freeze the result of any load out of the sandbox.

It would furthermore be ideal if we could run the flatbuffers parsing code directly on the guest memory, in order to avoid unnecessary copies. That is unfortunately probably not viable at the present time: because the generated flatbuffers parsing code doesn’t use atomic or volatile accesses, it is likely to introduce double-read vulnerabilities.

In short, none of the Rust-level operations available to us do the right thing, at the Rust spec level or the LLVM spec level. Our major remaining options are therefore:

• Choose one of the options that is available to us, and accept that we are doing something unsound according to the spec, but hope that no reasonable compiler could possibly notice.
• Use inline assembly per architecture, for which we would only need to worry about the architecture’s memory model (which is far less demanding).

The leading candidate for the first option would seem to be to simply use volatile accesses; there seems to be wide agreement that this should be a valid use case for them (even if it isn’t now), and projects like Linux and rust-vmm already use C11 volatile for this purpose. It is also worth noting that because we still do need to synchronize with the guest when it is being well-behaved, we would ideally use volatile acquire loads and volatile release stores for interacting with the stack pointer in the guest in this case. Unfortunately, while those operations are defined in LLVM, they are not presently exposed to Rust. While atomic fences that are not associated with memory accesses (std::sync::atomic::fence) might at first glance seem to help with this problem, they unfortunately do not [6]:

A fence ‘A’ which has (at least) Release ordering semantics, synchronizes with a fence ‘B’ with (at least) Acquire semantics, if and only if there exist operations X and Y, both operating on some atomic object ‘M’ such that A is sequenced before X, Y is sequenced before B and Y observes the change to M. This provides a happens-before dependence between A and B.

Note that the X and Y must be to an atomic object.

We consequently assume that there has been a strong architectural fence on a vmenter/vmexit between data being read and written. This is unsafe (not guaranteed in the type system)!

[1] N3047 C23 Working Draft. https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3047.pdf [2] N4950 C++23 Working Draft. https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2023/n4950.pdf [3] LLVM Language Reference Manual, Memory Model for Concurrent Operations. https://llvm.org/docs/LangRef.html#memmodel [4] P1152R0: Deprecating volatile. JF Bastien. https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p1152r0.html [5] P1382R1: volatile_load<T> and volatile_store<T>. JF Bastien, Paul McKenney, Jeffrey Yasskin, and the indefatigable TBD. https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1382r1.pdf [6] Documentation for std::sync::atomic::fence. https://doc.rust-lang.org/std/sync/atomic/fn.fence.html

Implementations

impl HostSharedMemory

pub fn read<T: AllValid>(&self, offset: usize) -> Result<T>

Read a value of type T, whose representation is the same between the sandbox and the host, and which has no invalid bit patterns

pub fn write<T: AllValid>(&self, offset: usize, data: T) -> Result<()>

Write a value of type T, whose representation is the same between the sandbox and the host, and which has no invalid bit patterns

pub fn copy_to_slice(&self, slice: &mut [u8], offset: usize) -> Result<()>

Copy the contents of the slice into the sandbox at the specified offset

pub fn copy_from_slice(&self, slice: &[u8], offset: usize) -> Result<()>

Copy the contents of the sandbox at the specified offset into the slice

pub fn fill(&mut self, value: u8, offset: usize, len: usize) -> Result<()>

Fill the memory in the range [offset, offset + len) with value

pub fn push_buffer( &mut self, buffer_start_offset: usize, buffer_size: usize, data: &[u8], ) -> Result<()>

Pushes the given data onto shared memory to the buffer at the given offset. NOTE! buffer_start_offset must point to the beginning of the buffer

pub fn try_pop_buffer_into<T>( &mut self, buffer_start_offset: usize, buffer_size: usize, ) -> Result<T>

where T: for<'b> TryFrom<&'b [u8]>,

Pops the given given buffer into a T and returns it. NOTE! the data must be a size-prefixed flatbuffer, and buffer_start_offset must point to the beginning of the buffer

Trait Implementations

impl Clone for HostSharedMemory

fn clone(&self) -> HostSharedMemory

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for HostSharedMemory

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl SharedMemory for HostSharedMemory

fn region(&self) -> &HostMapping

Return a readonly reference to the host mapping backing this SharedMemory

fn with_exclusivity<T, F: FnOnce(&mut ExclusiveSharedMemory) -> T>( &mut self, f: F, ) -> Result<T>

Run some code with exclusive access to the SharedMemory underlying this. If the SharedMemory is not an ExclusiveSharedMemory, any concurrent accesses to the relevant HostSharedMemory/GuestSharedMemory may make this fail, or be made to fail by this, and should be avoided.

fn base_addr(&self) -> usize

Return the base address of the host mapping of this region. Following the general Rust philosophy, this does not need to be marked as unsafe because doing anything with this pointer itself requires unsafe.

fn base_ptr(&self) -> *mut u8

Return the base address of the host mapping of this region as a pointer. Following the general Rust philosophy, this does not need to be marked as unsafe because doing anything with this pointer itself requires unsafe.

fn mem_size(&self) -> usize

Return the length of usable memory contained in self. The returned size does not include the size of the surrounding guard pages.

fn raw_ptr(&self) -> *mut u8

Return the raw base address of the host mapping, including the guard pages.

fn raw_mem_size(&self) -> usize

Return the raw size of the host mapping, including the guard pages.

impl Send for HostSharedMemory