Struct BytesBuf 

pub struct BytesBuf { /* private fields */ }

Assembles byte sequences, exposing them as BytesViews.

The buffer owns some memory capacity into which it allows you to write a sequence of bytes that you can thereafter extract as one or more BytesViews over immutable data. Mutation of the buffer contents is append-only - once data has been written into the buffer, it cannot be modified.

Capacity must be reserved in advance (e.g. via reserve()) before you can write data into the buffer. The exception to this is when appending an existing BytesView via put_bytes() because appending a BytesView is a zero-copy operation that reuses the view’s existing memory capacity.

Memory capacity

A single BytesBuf can use memory capacity from any memory provider, including a mix of different memory providers. All methods that extend the memory capacity require the caller to provide a reference to the memory provider to use.

To understand how to obtain access to a memory provider, see Producing Byte Sequences.

When data is extracted from the buffer by consuming it (via consume() or consume_all()), ownership of the used memory capacity is transferred to the returned BytesView. Any leftover memory capacity remains in the buffer, ready to receive further writes.

Conceptual design

The memory capacity owned by a BytesBuf can be viewed as two regions:

• Filled memory - data has been written into this memory but this data has not yet been consumed as a BytesView. Nevertheless, this data may already be in use because it may have been exposed via peek(), which does not consume it from the buffer. Memory capacity is removed from this region when bytes are consumed from the buffer.
• Available memory - no data has been written into this memory. Calling any of the write methods on BytesBuf will write data to the start of this region and transfer the affected capacity to the filled memory region.

Existing BytesViews can be appended to the BytesBuf via put_bytes() without consuming capacity as each appended BytesView brings its own backing memory capacity.

Memory layout

A byte sequence represented by bytesbuf types may consist of any number of separate byte slices, each of which contains one or more bytes of data.

There is no upper or lower bound on the length of each slice of bytes. At one extreme, a byte sequence may be entirely represented by a single slice of bytes. At the opposite extreme, it is legal for each byte to be represented by a separate non-consecutive slice.

Examples of legal memory layouts for the byte sequence b'Hello':

• ['H', 'e', 'l', 'l', 'o']
• ['H', 'e'], ['l', 'l', 'o']
• ['H'], ['e'], ['l'], ['l'], ['o']

Code using these APIs is required to work with any memory layout, as there are no guarantees on which layout will be used for which byte sequences.

Example

use bytesbuf::BytesBuf;

const HEADER_MAGIC: &[u8] = b"HDR\x00";

let mut buf = memory.reserve(64);

// Build a message from various pieces.
buf.put_slice(HEADER_MAGIC);
buf.put_num_be(1_u16); // Version
buf.put_num_be(42_u32); // Payload length
buf.put_num_be(0xDEAD_BEEF_u64); // Checksum

// Consume the buffered data as an immutable BytesView.
let message = buf.consume_all();
assert_eq!(message.len(), 18);

Implementations

impl BytesBuf

pub fn new() -> Self

Creates an instance without any memory capacity.

pub fn from_blocks<I>(blocks: I) -> Self

where I: IntoIterator<Item = Block>,

Creates an instance that owns the provided memory blocks.

This is the API used by memory providers to issue rented memory capacity to callers. Unless you are implementing a memory provider, you will not need to call this function. Instead, use either Memory::reserve() or BytesBuf::reserve().

Blocks are unordered

There is no guarantee that the BytesBuf uses the blocks in the order provided to this function. Blocks may be used in any order.

pub fn reserve( &mut self, additional_bytes: usize, memory_provider: &impl Memory, )

Adds enough memory capacity to accommodate at least additional_bytes of content.

After this call, remaining_capacity() will be at least additional_bytes.

The memory provider may provide more capacity than requested - additional_bytes is only a lower bound.

Example

use bytesbuf::BytesBuf;

let mut buf = BytesBuf::new();

// Must reserve capacity before writing.
buf.reserve(16, &memory);
assert!(buf.remaining_capacity() >= 16);

buf.put_num_be(0x1234_5678_u32);

// Can reserve more capacity at any time.
buf.reserve(100, &memory);
assert!(buf.remaining_capacity() >= 100);

Panics

Panics if the resulting total buffer capacity would be greater than usize::MAX.

pub fn peek(&self) -> BytesView

Peeks at the contents of the filled bytes region.

The returned BytesView covers all data in the buffer but does not consume any of the data.

Functionally similar to consume_all() except all the data remains in the buffer and can still be consumed later.

Example

use bytesbuf::mem::Memory;

let mut buf = memory.reserve(16);
buf.put_num_be(0x1234_u16);
buf.put_num_be(0x5678_u16);

// Peek at the data without consuming it.
let mut peeked = buf.peek();
assert_eq!(peeked.get_num_be::<u16>(), 0x1234);
assert_eq!(peeked.get_num_be::<u16>(), 0x5678);

// Despite consuming from peeked, the buffer still contains all data.
assert_eq!(buf.len(), 4);

let consumed = buf.consume_all();
assert_eq!(consumed.len(), 4);

pub fn len(&self) -> usize

How many bytes of data are in the buffer, ready to be consumed.

Example

use bytesbuf::mem::Memory;

let mut buf = memory.reserve(32);
assert_eq!(buf.len(), 0);

buf.put_num_be(0x1234_5678_u32);
assert_eq!(buf.len(), 4);

buf.put_slice(*b"Hello");
assert_eq!(buf.len(), 9);

_ = buf.consume(4);
assert_eq!(buf.len(), 5);

pub fn is_empty(&self) -> bool

Whether the buffer is empty (contains no data).

This does not imply that the buffer has no remaining memory capacity.

pub fn capacity(&self) -> usize

The total capacity of the buffer.

This is the total length of the filled bytes and the available bytes regions.

Example

use bytesbuf::BytesBuf;

let mut buf = BytesBuf::new();
assert_eq!(buf.capacity(), 0);

buf.reserve(100, &memory);
let initial_capacity = buf.capacity();
assert!(initial_capacity >= 100);

// Writing does not change capacity.
buf.put_slice(*b"Hello");
assert_eq!(buf.capacity(), initial_capacity);

// Consuming reduces capacity (memory is transferred to the BytesView).
_ = buf.consume(5);
assert!(buf.capacity() < initial_capacity);

pub fn remaining_capacity(&self) -> usize

How many more bytes can be written into the buffer before its memory capacity is exhausted.

Example

use bytesbuf::BytesBuf;

let mut buf = BytesBuf::new();

buf.reserve(100, &memory);
let initial_remaining = buf.remaining_capacity();
assert!(initial_remaining >= 100);

// Writing reduces remaining capacity.
buf.put_slice(*b"Hello");
assert_eq!(buf.remaining_capacity(), initial_remaining - 5);

// Reserving more increases remaining capacity.
buf.reserve(200, &memory);
assert!(buf.remaining_capacity() >= 200);

// Consuming buffered data does NOT affect remaining capacity.
let remaining_before_consume = buf.remaining_capacity();
_ = buf.consume(5);
assert_eq!(buf.remaining_capacity(), remaining_before_consume);

pub fn consume(&mut self, len: usize) -> BytesView

Consumes len bytes from the beginning of the buffer.

The consumed bytes and the memory capacity that backs them are removed from the buffer.

Example

use bytesbuf::mem::Memory;

let mut buf = memory.reserve(32);

buf.put_num_be(0x1111_u16);
buf.put_num_be(0x2222_u16);

// Consume first part.
let mut first = buf.consume(2);
assert_eq!(first.get_num_be::<u16>(), 0x1111);

// Write more data.
buf.put_num_be(0x3333_u16);

// Consume remaining data.
let mut rest = buf.consume(4);
assert_eq!(rest.get_num_be::<u16>(), 0x2222);
assert_eq!(rest.get_num_be::<u16>(), 0x3333);

Panics

Panics if the buffer does not contain at least len bytes.

pub fn consume_checked(&mut self, len: usize) -> Option<BytesView>

Consumes len bytes from the beginning of the buffer.

Returns None if the buffer does not contain at least len bytes.

The consumed bytes and the memory capacity that backs them are removed from the buffer.

pub fn consume_all(&mut self) -> BytesView

Consumes all bytes in the buffer.

The consumed bytes and the memory capacity that backs them are removed from the buffer.

Example

use bytesbuf::mem::Memory;

let mut buf = memory.reserve(32);
buf.put_slice(*b"Hello, ");
buf.put_slice(*b"world!");
buf.put_num_be(0x2121_u16); // "!!"

let message = buf.consume_all();

assert_eq!(message, b"Hello, world!!!");
assert!(buf.is_empty());

pub fn first_unfilled_slice(&mut self) -> &mut [MaybeUninit<u8>]

The first slice of memory in the remaining capacity of the buffer.

This allows you to manually write into the buffer instead of using the various provided convenience methods. Only the first slice of the remaining capacity is exposed at any given time by this API.

After writing data to the start of this slice, call advance() to indicate how many bytes have been filled with data. The next call to first_unfilled_slice() will return the next slice of memory you can write into. This slice must be completely filled before the next slice is exposed (a partial fill will simply return the remaining range from the same slice in the next call).

To write to multiple slices concurrently, use begin_vectored_write().

Memory layout

A byte sequence represented by bytesbuf types may consist of any number of separate byte slices, each of which contains one or more bytes of data.

There is no upper or lower bound on the length of each slice of bytes. At one extreme, a byte sequence may be entirely represented by a single slice of bytes. At the opposite extreme, it is legal for each byte to be represented by a separate non-consecutive slice.

Examples of legal memory layouts for the byte sequence b'Hello':

• ['H', 'e', 'l', 'l', 'o']
• ['H', 'e'], ['l', 'l', 'o']
• ['H'], ['e'], ['l'], ['l'], ['o']

Code using these APIs is required to work with any memory layout, as there are no guarantees on which layout will be used for which byte sequences.

Example

use bytesbuf::mem::Memory;

let mut buf = memory.reserve(64);
let data_to_write: &[u8] = b"0123456789";

// Write data without assuming the length of first_unfilled_slice().
let mut written = 0;

while written < data_to_write.len() {
    let dst = buf.first_unfilled_slice();

    let bytes_to_write = dst.len().min(data_to_write.len() - written);

    for i in 0..bytes_to_write {
        dst[i].write(data_to_write[written + i]);
    }

    // SAFETY: We just initialized `bytes_to_write` bytes.
    unsafe {
        buf.advance(bytes_to_write);
    }
    written += bytes_to_write;
}

assert_eq!(buf.consume_all(), b"0123456789");

pub unsafe fn advance(&mut self, count: usize)

Signals that count bytes have been written to the start of first_unfilled_slice().

The next call to first_unfilled_slice() will return the next slice of memory that can be filled with data.

Example

use bytesbuf::mem::Memory;

let mut buf = memory.reserve(64);
let data_to_write: &[u8] = b"0123456789";

// Write data without assuming the length of first_unfilled_slice().
let mut written = 0;

while written < data_to_write.len() {
    let dst = buf.first_unfilled_slice();

    let bytes_to_write = dst.len().min(data_to_write.len() - written);

    for i in 0..bytes_to_write {
        dst[i].write(data_to_write[written + i]);
    }

    // SAFETY: We just initialized `bytes_to_write` bytes.
    unsafe {
        buf.advance(bytes_to_write);
    }
    written += bytes_to_write;
}

assert_eq!(buf.consume_all(), b"0123456789");

Safety

The caller must guarantee that count bytes from the beginning of first_unfilled_slice() have been initialized.

pub fn begin_vectored_write( &mut self, max_len: Option<usize>, ) -> BytesBufVectoredWrite<'_>

Concurrently writes data into all the byte slices that make up the buffer.

The vectored write takes exclusive ownership of the buffer for the duration of the operation and allows individual slices of the remaining capacity to be filled concurrently, up to an optional limit of max_len bytes.

Some I/O operations are naturally limited to a maximum number of bytes that can be transferred, so the length limit here allows you to project a restricted view of the available capacity without having to limit the true capacity of the buffer.

Example

use std::ptr;

use bytesbuf::mem::Memory;

let mut buf = memory.reserve(64);
let capacity = buf.remaining_capacity();

let mut vectored = buf.begin_vectored_write(None);
let mut slices: Vec<_> = vectored.iter_slices_mut().collect();

// Fill all slices with 0xAE bytes.
// In practice, these could be filled concurrently by vectored I/O APIs.
let mut total_written = 0;
for slice in &mut slices {
    // SAFETY: Writing valid u8 values to the entire slice.
    unsafe {
        ptr::write_bytes(slice.as_mut_ptr(), 0xAE, slice.len());
    }
    total_written += slice.len();
}

// SAFETY: We initialized `total_written` bytes sequentially.
unsafe {
    vectored.commit(total_written);
}

assert_eq!(buf.len(), capacity);

Panics

Panics if max_len is greater than the remaining capacity of the buffer.

pub fn begin_vectored_write_checked( &mut self, max_len: Option<usize>, ) -> Option<BytesBufVectoredWrite<'_>>

Concurrently writes data into all the byte slices that make up the buffer.

The vectored write takes exclusive ownership of the buffer for the duration of the operation and allows individual slices of the remaining capacity to be filled concurrently, up to an optional limit of max_len bytes.

Some I/O operations are naturally limited to a maximum number of bytes that can be transferred, so the length limit here allows you to project a restricted view of the available capacity without having to limit the true capacity of the buffer.

Returns

Returns None if max_len is greater than the remaining capacity of the buffer.

pub fn extend_lifetime(&self) -> MemoryGuard

Extends the lifetime of the memory capacity backing this buffer.

This can be useful when unsafe code is used to reference the contents of a BytesBuf and it is possible to reach a condition where the BytesBuf itself no longer exists, even though the contents are referenced (e.g. because the remaining references are in non-Rust code).

pub fn as_write<M: Memory>(&mut self, memory: &M) -> impl Write

Exposes the instance through the Write trait.

The memory capacity of the BytesBuf will be automatically extended on demand with additional capacity from the supplied memory provider.

Example

use std::io::Write;

use bytesbuf::mem::Memory;

let mut buf = memory.reserve(32);
{
    let mut writer = buf.as_write(&memory);
    writer.write_all(b"Hello, ")?;
    writer.write_all(b"world!")?;
}

assert_eq!(buf.consume_all(), b"Hello, world!");

impl BytesBuf

pub fn put_slice(&mut self, src: impl Borrow<[u8]>)

Appends a slice of bytes to the buffer.

Example

use bytesbuf::mem::Memory;

let mut buf = memory.reserve(32);

buf.put_slice(*b"Hello, ");
buf.put_slice(*b"world!");

assert_eq!(buf.consume_all(), b"Hello, world!");

Panics

Panics if there is insufficient remaining capacity in the buffer.

pub fn put_bytes(&mut self, view: BytesView)

Appends a byte sequence to the buffer.

This reuses the existing capacity of the view being appended.

Example

use bytesbuf::BytesView;
use bytesbuf::mem::Memory;

let mut buf = memory.reserve(16);

// Create a view with its own memory.
let header = BytesView::copied_from_slice(b"HDR", &memory);

buf.put_slice(*b"data");
buf.put_bytes(header); // Zero-copy append

assert_eq!(buf.consume_all(), b"dataHDR");

Panics

Panics if there is insufficient remaining capacity in the buffer.

pub fn put_byte(&mut self, value: u8)

Appends a u8 to the buffer.

Example

use bytesbuf::mem::Memory;

let mut buf = memory.reserve(8);

buf.put_byte(0xCA);
buf.put_byte(0xFE);

assert_eq!(buf.consume_all(), &[0xCA, 0xFE]);

Panics

Panics if there is insufficient remaining capacity in the buffer.

pub fn put_byte_repeated(&mut self, value: u8, count: usize)

Appends multiple repetitions of a u8 to the buffer.

Example

use bytesbuf::mem::Memory;

let mut buf = memory.reserve(16);

// Write a header followed by zero-padding.
buf.put_slice(*b"HDR:");
buf.put_byte_repeated(0x00, 12);

let data = buf.consume_all();
assert_eq!(data.len(), 16);
assert_eq!(
    data.first_slice(),
    b"HDR:\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
);

Panics

Panics if there is insufficient remaining capacity in the buffer.

pub fn put_num_le<T: ToBytes>(&mut self, value: T)

Appends a number of type T in little-endian representation to the buffer.

Example

use bytesbuf::mem::Memory;

let mut buf = memory.reserve(16);

buf.put_num_le(0x1234_u16);
buf.put_num_le(0xDEAD_BEEF_u32);

let data = buf.consume_all();
// Little-endian: least significant byte first.
assert_eq!(data.first_slice(), &[0x34, 0x12, 0xEF, 0xBE, 0xAD, 0xDE]);

Panics

Panics if there is insufficient remaining capacity in the buffer.

pub fn put_num_be<T: ToBytes>(&mut self, value: T)

Appends a number of type T in big-endian representation to the buffer.

Example

use bytesbuf::mem::Memory;

let mut buf = memory.reserve(16);

buf.put_num_be(0xCAFE_u16);
buf.put_num_be(0xBABE_u16);

let data = buf.consume_all();
// Big-endian: most significant byte first.
assert_eq!(data, &[0xCA, 0xFE, 0xBA, 0xBE]);

Panics

Panics if there is insufficient remaining capacity in the buffer.

pub fn put_num_ne<T: ToBytes>(&mut self, value: T)

Appends a number of type T in native-endian representation to the buffer.

Example

use bytesbuf::mem::Memory;

let mut buf = memory.reserve(16);

buf.put_num_ne(0xABCD_u16);
buf.put_num_ne(0x1234_5678_9ABC_DEF0_u64);

let mut view = buf.consume_all();
// Reading back in native-endian gives the original values.
assert_eq!(view.get_num_ne::<u16>(), 0xABCD);
assert_eq!(view.get_num_ne::<u64>(), 0x1234_5678_9ABC_DEF0);

Panics

Panics if there is insufficient remaining capacity in the buffer.

Trait Implementations

impl BufMut for BytesBuf

Available on crate features bytes-compat only.

fn remaining_mut(&self) -> usize

Returns the number of bytes that can be written from the current position until the end of the buffer is reached.

unsafe fn advance_mut(&mut self, cnt: usize)

Advance the internal cursor of the BufMut

fn chunk_mut(&mut self) -> &mut UninitSlice

Returns a mutable slice starting at the current BufMut position and of length between 0 and BufMut::remaining_mut(). Note that this can be shorter than the whole remainder of the buffer (this allows non-continuous implementation).

fn has_remaining_mut(&self) -> bool

Returns true if there is space in self for more bytes.

fn put<T>(&mut self, src: T)

where T: Buf, Self: Sized,

Transfer bytes into self from src and advance the cursor by the number of bytes written.

fn put_slice(&mut self, src: &[u8])

Transfer bytes into self from src and advance the cursor by the number of bytes written.

fn put_bytes(&mut self, val: u8, cnt: usize)

Put cnt bytes val into self.

fn put_u8(&mut self, n: u8)

Writes an unsigned 8 bit integer to self.

fn put_i8(&mut self, n: i8)

Writes a signed 8 bit integer to self.

fn put_u16(&mut self, n: u16)

Writes an unsigned 16 bit integer to self in big-endian byte order.

fn put_u16_le(&mut self, n: u16)

Writes an unsigned 16 bit integer to self in little-endian byte order.

fn put_u16_ne(&mut self, n: u16)

Writes an unsigned 16 bit integer to self in native-endian byte order.

fn put_i16(&mut self, n: i16)

Writes a signed 16 bit integer to self in big-endian byte order.

fn put_i16_le(&mut self, n: i16)

Writes a signed 16 bit integer to self in little-endian byte order.

fn put_i16_ne(&mut self, n: i16)

Writes a signed 16 bit integer to self in native-endian byte order.

fn put_u32(&mut self, n: u32)

Writes an unsigned 32 bit integer to self in big-endian byte order.

fn put_u32_le(&mut self, n: u32)

Writes an unsigned 32 bit integer to self in little-endian byte order.

fn put_u32_ne(&mut self, n: u32)

Writes an unsigned 32 bit integer to self in native-endian byte order.

fn put_i32(&mut self, n: i32)

Writes a signed 32 bit integer to self in big-endian byte order.

fn put_i32_le(&mut self, n: i32)

Writes a signed 32 bit integer to self in little-endian byte order.

fn put_i32_ne(&mut self, n: i32)

Writes a signed 32 bit integer to self in native-endian byte order.

fn put_u64(&mut self, n: u64)

Writes an unsigned 64 bit integer to self in the big-endian byte order.

fn put_u64_le(&mut self, n: u64)

Writes an unsigned 64 bit integer to self in little-endian byte order.

fn put_u64_ne(&mut self, n: u64)

Writes an unsigned 64 bit integer to self in native-endian byte order.

fn put_i64(&mut self, n: i64)

Writes a signed 64 bit integer to self in the big-endian byte order.

fn put_i64_le(&mut self, n: i64)

Writes a signed 64 bit integer to self in little-endian byte order.

fn put_i64_ne(&mut self, n: i64)

Writes a signed 64 bit integer to self in native-endian byte order.

fn put_u128(&mut self, n: u128)

Writes an unsigned 128 bit integer to self in the big-endian byte order.

fn put_u128_le(&mut self, n: u128)

Writes an unsigned 128 bit integer to self in little-endian byte order.

fn put_u128_ne(&mut self, n: u128)

Writes an unsigned 128 bit integer to self in native-endian byte order.

fn put_i128(&mut self, n: i128)

Writes a signed 128 bit integer to self in the big-endian byte order.

fn put_i128_le(&mut self, n: i128)

Writes a signed 128 bit integer to self in little-endian byte order.

fn put_i128_ne(&mut self, n: i128)

Writes a signed 128 bit integer to self in native-endian byte order.

fn put_uint(&mut self, n: u64, nbytes: usize)

Writes an unsigned n-byte integer to self in big-endian byte order.

fn put_uint_le(&mut self, n: u64, nbytes: usize)

Writes an unsigned n-byte integer to self in the little-endian byte order.

fn put_uint_ne(&mut self, n: u64, nbytes: usize)

Writes an unsigned n-byte integer to self in the native-endian byte order.

fn put_int(&mut self, n: i64, nbytes: usize)

Writes low nbytes of a signed integer to self in big-endian byte order.

fn put_int_le(&mut self, n: i64, nbytes: usize)

Writes low nbytes of a signed integer to self in little-endian byte order.

fn put_int_ne(&mut self, n: i64, nbytes: usize)

Writes low nbytes of a signed integer to self in native-endian byte order.

fn put_f32(&mut self, n: f32)

Writes an IEEE754 single-precision (4 bytes) floating point number to self in big-endian byte order.

fn put_f32_le(&mut self, n: f32)

Writes an IEEE754 single-precision (4 bytes) floating point number to self in little-endian byte order.

fn put_f32_ne(&mut self, n: f32)

Writes an IEEE754 single-precision (4 bytes) floating point number to self in native-endian byte order.

fn put_f64(&mut self, n: f64)

Writes an IEEE754 double-precision (8 bytes) floating point number to self in big-endian byte order.

fn put_f64_le(&mut self, n: f64)

Writes an IEEE754 double-precision (8 bytes) floating point number to self in little-endian byte order.

fn put_f64_ne(&mut self, n: f64)

Writes an IEEE754 double-precision (8 bytes) floating point number to self in native-endian byte order.

fn limit(self, limit: usize) -> Limit<Self>

where Self: Sized,

Creates an adaptor which can write at most limit bytes to self.

fn writer(self) -> Writer<Self>

where Self: Sized,

Available on crate feature std only.

Creates an adaptor which implements the Write trait for self.

fn chain_mut<U>(self, next: U) -> Chain<Self, U>

where U: BufMut, Self: Sized,

Creates an adapter which will chain this buffer with another.

impl Debug for BytesBuf

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for BytesBuf

fn default() -> BytesBuf

Returns the “default value” for a type.

impl From<BytesView> for BytesBuf

fn from(value: BytesView) -> Self

Converts to this type from the input type.