Crate vortex_buffer

A library for working with custom aligned buffers of sized values.

The vortex-buffer crate is built around bytes::Bytes and therefore supports zero-copy cloning and slicing, but differs in that it can define and maintain a custom alignment.

• Buffer<T> and BufferMut<T> provide immutable and mutable wrappers around bytes::Bytes and bytes::BytesMut respectively.
• ByteBuffer and ByteBufferMut are type aliases for u8 buffers.
• BufferString is a wrapper around a ByteBuffer that enforces utf-8 encoding.
• ConstBuffer<T, const A: usize> provides similar functionality to Buffer<T> except with a compile-time alignment of A.
• buffer! and buffer_mut! macros with the same syntax as the builtin vec! macro for inline construction of buffers.

You can think of BufferMut<T> as similar to a Vec<T>, except that any operation that may cause a re-allocation, e.g. extend, will ensure the new allocation maintains the buffer’s defined alignment.

For example, it’s possible to incrementally build a Buffer<T> with a 4KB alignment.

use vortex_buffer::{Alignment, BufferMut};

let mut buf = BufferMut::<i32>::empty_aligned(Alignment::new(4096));
buf.extend(0i32..1_000);
assert_eq!(buf.as_ptr().align_offset(4096), 0)

Comparison

Implementation	Zero-copy	Custom Alignment	Typed
vortex_buffer::Buffer<T>	✔️	✔️	✔️
arrow_buffer::ScalarBuffer<T>	✔️	❌️️️	✔️
bytes::Bytes	✔️	❌️️️	❌️️️
Vec<T>	❌️	❌️️	✔️

Features

The arrow feature can be enabled to provide conversion functions to/from Arrow Rust buffers, including arrow_buffer::Buffer, arrow_buffer::ScalarBuffer<T>, and arrow_buffer::OffsetBuffer.

Macros

buffer

A macro for constructing buffers akin to vec![..].

buffer_mut

A macro for constructing buffers akin to vec![..].

Structs

Alignment

The alignment of a buffer.

Buffer

An immutable buffer of items of T.

BufferIterator

Owned iterator over a Buffer.

BufferMut

A mutable buffer that maintains a runtime-defined alignment through resizing operations.

BufferString

A wrapper around a ByteBuffer that guarantees that the buffer contains valid UTF-8.

ConstBuffer

A buffer of items of T with a compile-time alignment.

Traits

AlignedBuf

An extension to the Buf trait that provides a function copy_to_aligned similar to copy_to_bytes that allows for zero-copy aligned reads where possible.

Type Aliases

ByteBuffer

An immutable buffer of u8.

ByteBufferMut

A mutable buffer of u8.

ConstByteBuffer

A const-aligned buffer of u8.