Crate vortex_buffer

Expand description

A library for working with custom aligned buffers of sized values.

The vortex-buffer crate is built around bytes::Bytes and therefore supports zero-copy cloning and slicing, but differs in that it can define and maintain a custom alignment.

Buffer<T> and BufferMut<T> provide immutable and mutable wrappers around bytes::Bytes and bytes::BytesMut respectively.
ByteBuffer and ByteBufferMut are type aliases for u8 buffers.
BufferString is a wrapper around a ByteBuffer that enforces utf-8 encoding.
ConstBuffer<T, const A: usize> provides similar functionality to Buffer<T> except with a compile-time alignment of A.
buffer! and buffer_mut! macros with the same syntax as the builtin vec! macro for inline construction of buffers.

You can think of BufferMut<T> as similar to a Vec<T>, except that any operation that may cause a re-allocation, e.g. extend, will ensure the new allocation maintains the buffer’s defined alignment.

For example, it’s possible to incrementally build a Buffer<T> with a 4KB alignment.

use vortex_buffer::{Alignment, BufferMut};

let mut buf = BufferMut::<i32>::empty_aligned(Alignment::new(4096));
buf.extend(0i32..1_000);
assert_eq!(buf.as_ptr().align_offset(4096), 0)

§Comparison

Implementation	Zero-copy	Custom Alignment	Typed
`vortex_buffer::Buffer<T>`	✔️	✔️	✔️
`arrow_buffer::ScalarBuffer<T>`	✔️	❌️️️	✔️
`bytes::Bytes`	✔️	❌️️️	❌️️️
`Vec<T>`	❌️	❌️️	✔️

§Features

The arrow feature can be enabled to provide conversion functions to/from Arrow Rust buffers, including arrow_buffer::Buffer, arrow_buffer::ScalarBuffer<T>, and arrow_buffer::OffsetBuffer.

Macros§

bitbuffer: A macro for constructing bit-buffers akin to vec![..].
bitbuffer_mut: A macro for constructing bit-buffers akin to vec![..].
buffer: A macro for constructing buffers akin to vec![..].
buffer_mut: A macro for constructing buffers akin to vec![..].

Structs§

Alignment: The alignment of a buffer.
BitBuffer: An immutable bitset stored as a packed byte buffer.
BitBufferMut: A mutable bitset buffer that allows random access to individual bits for set and get.
BitChunkIterator: Iterator over chunks of 64 bits represented as an u64
BitChunks: Iterates over an arbitrarily aligned byte buffer
BitIndexIterator: An iterator of usize whose index in a provided bitmask is true
BitIterator: Iterator over the bits within a packed bitmask
BitSliceIterator: Iterator of contiguous ranges of set bits within a provided packed bitmask
Buffer: An immutable buffer of items of T.
BufferIterator: Owned iterator over a Buffer.
BufferMut: A mutable buffer that maintains a runtime-defined alignment through resizing operations.
BufferString: A wrapper around a ByteBuffer that guarantees that the buffer contains valid UTF-8.
ConstBuffer: A buffer of items of T with a compile-time alignment.
Iter: An iterator over Buffer elements.
UnalignedBitChunk: Iterates over an arbitrarily aligned byte buffer

Traits§

AlignedBuf: An extension to the Buf trait that provides a function copy_to_aligned similar to copy_to_bytes that allows for zero-copy aligned reads where possible.

Type Aliases§

ByteBuffer: An immutable buffer of u8.
ByteBufferMut: A mutable buffer of u8.
ConstByteBuffer: A const-aligned buffer of u8.
UnalignedBitChunkIterator: Iterator over an UnalignedBitChunk