Crate presser 

🗜 presser

Utilities to help make copying data around into raw, possibly-uninitialized buffers easier and safer.

Motivation

presser can help you when copying data into raw buffers. One primary use-case is copying data into graphics-api-allocated buffers which will then be accessed by the GPU. Common methods for doing this right now in Rust can often invoke UB in subtle and hard-to-see ways. For example, viewing an allocated but uninitialized buffer as an &mut [u8] is instantly undefined behavior*, and transmuteing even a T: Copy type which has any padding bytes in its layout as a &[u8] to be the source of a copy is also instantly undefined behavior, in both cases because it is invalid to create a reference to an invalid value (and uninitialized memory is an invalid u8), even if your code never actually accesses that memory. This immediately makes what seems like the most straightforward way to copy data into buffers unsound 😬.

presser helps with this by allowing you to view raw allocated memory of some size as a “Slab” of memory and then provides safe, valid ways to copy data into that memory. For example, you could implement Slab for your GPU-allocated buffer type, or use the built-in RawAllocation workflow described below, then use copy_to_offset_with_align to copy any T: Copy data into that buffer safely for use on the GPU. Of course, if your T doesn’t have the correct layout the GPU expects, accessing it on the GPU side may still be unsound or at least give an error.

* If you’re currently thinking to yourself “bah! what’s the issue? surely an uninit u8 is just any random bit pattern and that’s fine we don’t care,” check out this blog post by @RalfJung, one of the people leading the effort to better define Rust’s memory and execution model. As is explored in that blog post, an uninit piece of memory is not simply an arbitrary bit pattern, it is a wholly separate state about a piece of memory, outside of its value, which lets the compiler perform optimizations that reorder, delete, and otherwise change the actual execution flow of your program in ways that cannot be described simply by “the value could have some possible bit pattern”. LLVM and Clang are changing themselves to require special noundef attribute to perform many important optimizations that are otherwise unsound. For a concrete example of the sorts of problems this can cause, see this issue @scottmcm hit.

Introduction

The main idea is to implement Slab on raw-buffer-esque-types (see the Slab safety docs), which then enables the use of the other functions within the crate.

Depending on your use case, you may be able to implement Slab directly for your buffer type, or it may be more convenient or necessary to create a wrapping struct that borrows your raw buffer type and in turn implements Slab. For an example of this, see RawAllocation and BorrowedRawAllocation, which you may also use directly. The idea is to create a RawAllocation to your buffer, which you then borrow into a BorrowedRawAllocation (which implements Slab) by calling the unsafe function RawAllocation::borrow_as_slab

Once you have a slab, you can use the copy helper functions provided at the crate root, for example, copy_to_offset and copy_to_offset_with_align.

Example

#[derive(Clone, Copy)]
#[repr(C)]
struct MyDataStruct {
    a: u8,
    b: u32,
}

let my_data = MyDataStruct { a: 0, b: 42 };

// allocate an uninit buffer of some size
let my_buffer: MyBufferType = some_api.alloc_buffer_size(2048);

// use `RawAllocation` helper to allow access to a presser `Slab`.
// alternatively, you could implement the `Slab` on `MyByfferType` directly if that
// type is owned by your code!
let raw_allocation = presser::RawAllocation::from_raw_parts(my_buffer.ptr(), my_buffer.size());

// here we assert that we have exclusive access to the data in the buffer, and get the actual
// `Slab` to use to copy into.
let slab = unsafe { raw_allocation.borrow_as_slab(); }

// now we may safely copy `my_data` into `my_buffer`, starting at a minimum offset of 0 into the buffer
let copy_record = presser::copy_to_offset(&my_data, &mut slab, 0)?;

// note that due to alignment requirements of `my_data`, the *actual* start of the bytes of
// `my_data` may be placed at a different offset than requested. so, we check the returned
// `CopyRecord` to check the actual start offset of the copied data.
let actual_start_offset = copy_record.copy_start_offset;

#[no_std]

This crate supports no_std environments by building without the ‘std’ feature. This will limit some of the fuctions the crate can perform.

Safety

An important note is that obeying the safety rules specified in the Slab safety documentation only guarantees safety for the direct results of the copy operations performed by the helper functions exported at the crate root (and the safe functions on Slab). However, it is not guaranteed that operations which would previously have been safe to perform using same backing memory that the Slab you copied into used are still safe.

For example, say you have a fully-initialized chunk of bytes (like a Vec<u8>), which you view as a Slab, and then (safely) perform a copy operation into using copy_to_offset. If the T you copied into it has any padding bytes in its memory layout, then the memory locations where those padding bytes now exist in the underlying Vec’s memory must now be treated as uninitialized. As such, taking any view into that byte vector which relies on those newly-uninit bytes being initialized to be valid (for example, taking a &[u8] slice of the Vec which includes those bytes, even if your code never actually reads from that slice) is now instant undefined behavior.

Structs

BorrowedRawAllocation

Represents the unique borrow of a contiguous piece of a single allocation with some layout that is used as a data copying destination. May be wholly or partially uninitialized.

CopyRecord

Record of the results of a copy operation

RawAllocation

Represents a contiguous piece of a single allocation with some layout. May be wholly or partially uninitialized.

Enums

CopyError

An error that may occur during a copy operation.

Traits

Slab

Represents a contiguous piece of a single allocation with some layout that is used as a data copying destination. May be wholly or partially uninitialized.

Functions

clone_into_maybe_uninit_slice

Clones the elements from src to dst, returning a mutable reference to the now initialized contents of dst. Any already initialized elements will not be dropped.

copy_from_iter_to_offset_with_alignstd

Copies from src iterator into the memory represented by dst starting at a minimum location of start_offset bytes past the start of dst.

copy_from_iter_to_offset_with_align_packed

Like copy_from_iter_to_offset_with_align except that alignment between elements yielded by the iterator will ignore min_alignment and rather only be aligned to the alignment of T.

copy_from_slice_to_offset

Copies from slice into the memory represented by dst starting at a minimum location of start_offset bytes past the start of self.

copy_from_slice_to_offset_with_align

Copies from slice into the memory represented by dst starting at a minimum location of start_offset bytes past the start of dst.

copy_into_maybe_uninit_slice

Copies the elements from src to dst, returning a mutable reference to the now initialized contents of dst.

copy_to_offset

Copies src into the memory represented by dst starting at a minimum location of start_offset bytes past the start of dst.

copy_to_offset_with_align

Copies src into the memory represented by dst starting at a minimum location of start_offset bytes past the start of dst and with minimum alignment min_alignment.

maybe_uninit_slice_from_vec^⚠std

Takes a Vec and unsafely resizes it to the given length, returning a mutable slice to MaybeUninit<T> for each item in the newly-resized Vec.