Struct ChunkedBytes

pub struct ChunkedBytes { /* private fields */ }

A non-contiguous buffer for efficient serialization of data structures.

A ChunkedBytes container has a staging buffer to coalesce small byte sequences of source data, and a queue of byte chunks split off the staging buffer that can be incrementally consumed by an output API such as an object implementing AsyncWrite. Once the number of bytes in the staging buffer reaches a certain configured chunk size, the buffer content is split off to form a new chunk.

Unlike loosely::ChunkedBytes, this variant of the ChunkedBytes container never produces chunks larger than the configured size. This comes at a cost of increased processing overhead and sometimes more allocated memory needed to keep the buffered data, so the applications that don’t benefit from the strict limit should prefer loosely::ChunkedBytes.

Refer to the documentation on the methods available for ChunkedBytes, including the methods of traits Buf and BufMut, for details on working with this container.

Implementations

impl ChunkedBytes

pub fn new() -> Self

Creates a new ChunkedBytes container with the chunk size limit set to a default value.

pub fn with_chunk_size_limit(chunk_size: usize) -> Self

Creates a new ChunkedBytes container with the given chunk size limit.

pub fn with_profile(chunk_size: usize, chunking_capacity: usize) -> Self

The fully detailed constructor for ChunkedBytes. The chunk size limit is given in chunk_size, and an upper estimate of the number of chunks this container could be expected to have at any moment of time should be given in chunking_capacity. More chunks can still be held, but this may cause reallocations of internal data structures.

pub fn chunk_size_limit(&self) -> usize

Returns the size this ChunkedBytes container uses as the limit for splitting off complete chunks.

Note that the size of produced chunks may be smaller than the configured value, due to the allocation strategy used internally by the implementation and also depending on the pattern of usage.

pub fn is_empty(&self) -> bool

Returns true if the ChunkedBytes container has no complete chunks and the staging buffer is empty.

pub fn flush(&mut self)

Splits any bytes that are currently in the staging buffer into a new complete chunk. If the staging buffer is empty, this method does nothing.

Most users should not need to call this method. It is called internally when needed by the methods that advance the writing position.

pub fn put_bytes(&mut self, src: Bytes)

Appends a Bytes slice to the container without copying the data.

If src is empty, this method does nothing. Otherwise, if there are any bytes currently in the staging buffer, they are split to form a complete chunk. Next, src is appended as a sequence of chunks, split if necessary so that all chunks except the last are sized to the chunk size limit.

Performance Notes

For a small slice originating from a buffer that is not split or shared between other Bytes instances, copying the bytes with BufMut::put_slice may be faster than the overhead of atomic reference counting induced by use of this method.

pub fn drain_chunks(&mut self) -> DrainChunks<'_>

Returns an iterator that removes complete chunks from the ChunkedBytes container and yields the removed chunks as Bytes slice handles. This does not include bytes in the staging buffer.

The chunks are removed even if the iterator is dropped without being consumed until the end. It is unspecified how many chunks are removed if the DrainChunks value is not dropped, but the borrow it holds expires (e.g. due to std::mem::forget).

pub fn into_chunks(self) -> IntoChunks

Consumes the ChunkedBytes container to produce an iterator over its chunks. If there are bytes in the staging buffer, they are yielded as the last src.

The memory allocated for IntoChunks may be slightly more than the ChunkedBytes container it consumes. This is an infrequent side effect of making the internal state efficient in general for iteration.

Trait Implementations

impl Buf for ChunkedBytes

fn chunk(&self) -> &[u8]

Returns a slice of the bytes in the first extant complete chunk, or the bytes in the staging buffer if there are no unconsumed chunks.

It is more efficient to use chunks_vectored to gather all the disjoint slices for vectored output.

fn advance(&mut self, cnt: usize)

Advances the reading position by cnt, dropping the Bytes references to any complete chunks that the position has been advanced past and then advancing the starting position of the first remaining chunk. If there are no complete chunks left, the reading position is advanced in the staging buffer, effectively removing the consumed bytes.

Panics

This function may panic when cnt > self.remaining().

fn chunks_vectored<'a>(&'a self, dst: &mut [IoSlice<'a>]) -> usize

Fills dst sequentially with the slice views of the chunks, then the bytes in the staging buffer if any remain and there is another unfilled entry left in dst. Returns the number of IoSlice entries filled.

fn remaining(&self) -> usize

Returns the number of bytes between the current position and the end of the buffer.

fn has_remaining(&self) -> bool

Returns true if there are any more bytes to consume

fn copy_to_bytes(&mut self, len: usize) -> Bytes

Consumes len bytes inside self and returns new instance of Bytes with this data.

fn copy_to_slice(&mut self, dst: &mut [u8])

Copies bytes from self into dst.

fn get_u8(&mut self) -> u8

Gets an unsigned 8 bit integer from self.

fn get_i8(&mut self) -> i8

Gets a signed 8 bit integer from self.

fn get_u16(&mut self) -> u16

Gets an unsigned 16 bit integer from self in big-endian byte order.

fn get_u16_le(&mut self) -> u16

Gets an unsigned 16 bit integer from self in little-endian byte order.

fn get_i16(&mut self) -> i16

Gets a signed 16 bit integer from self in big-endian byte order.

fn get_i16_le(&mut self) -> i16

Gets a signed 16 bit integer from self in little-endian byte order.

fn get_u32(&mut self) -> u32

Gets an unsigned 32 bit integer from self in the big-endian byte order.

fn get_u32_le(&mut self) -> u32

Gets an unsigned 32 bit integer from self in the little-endian byte order.

fn get_i32(&mut self) -> i32

Gets a signed 32 bit integer from self in big-endian byte order.

fn get_i32_le(&mut self) -> i32

Gets a signed 32 bit integer from self in little-endian byte order.

fn get_u64(&mut self) -> u64

Gets an unsigned 64 bit integer from self in big-endian byte order.

fn get_u64_le(&mut self) -> u64

Gets an unsigned 64 bit integer from self in little-endian byte order.

fn get_i64(&mut self) -> i64

Gets a signed 64 bit integer from self in big-endian byte order.

fn get_i64_le(&mut self) -> i64

Gets a signed 64 bit integer from self in little-endian byte order.

fn get_u128(&mut self) -> u128

Gets an unsigned 128 bit integer from self in big-endian byte order.

fn get_u128_le(&mut self) -> u128

Gets an unsigned 128 bit integer from self in little-endian byte order.

fn get_i128(&mut self) -> i128

Gets a signed 128 bit integer from self in big-endian byte order.

fn get_i128_le(&mut self) -> i128

Gets a signed 128 bit integer from self in little-endian byte order.

fn get_uint(&mut self, nbytes: usize) -> u64

Gets an unsigned n-byte integer from self in big-endian byte order.

fn get_uint_le(&mut self, nbytes: usize) -> u64

Gets an unsigned n-byte integer from self in little-endian byte order.

fn get_int(&mut self, nbytes: usize) -> i64

Gets a signed n-byte integer from self in big-endian byte order.

fn get_int_le(&mut self, nbytes: usize) -> i64

Gets a signed n-byte integer from self in little-endian byte order.

fn get_f32(&mut self) -> f32

Gets an IEEE754 single-precision (4 bytes) floating point number from self in big-endian byte order.

fn get_f32_le(&mut self) -> f32

Gets an IEEE754 single-precision (4 bytes) floating point number from self in little-endian byte order.

fn get_f64(&mut self) -> f64

Gets an IEEE754 double-precision (8 bytes) floating point number from self in big-endian byte order.

fn get_f64_le(&mut self) -> f64

Gets an IEEE754 double-precision (8 bytes) floating point number from self in little-endian byte order.

fn take(self, limit: usize) -> Take<Self>

where Self: Sized,

Creates an adaptor which will read at most limit bytes from self.

fn chain<U>(self, next: U) -> Chain<Self, U>

where U: Buf, Self: Sized,

Creates an adaptor which will chain this buffer with another.

fn reader(self) -> Reader<Self>

where Self: Sized,

Creates an adaptor which implements the Read trait for self.

impl BufMut for ChunkedBytes

fn remaining_mut(&self) -> usize

Returns the number of bytes that can be written from the current position until the end of the buffer is reached.

unsafe fn advance_mut(&mut self, cnt: usize)

Advance the internal cursor of the BufMut

fn chunk_mut(&mut self) -> &mut UninitSlice

Returns a mutable slice starting at the current BufMut position and of length between 0 and BufMut::remaining_mut(). Note that this can be shorter than the whole remainder of the buffer (this allows non-continuous implementation).

fn has_remaining_mut(&self) -> bool

Returns true if there is space in self for more bytes.

fn put<T>(&mut self, src: T)

where T: Buf, Self: Sized,

Transfer bytes into self from src and advance the cursor by the number of bytes written.

fn put_slice(&mut self, src: &[u8])

Transfer bytes into self from src and advance the cursor by the number of bytes written.

fn put_u8(&mut self, n: u8)

Writes an unsigned 8 bit integer to self.

fn put_i8(&mut self, n: i8)

Writes a signed 8 bit integer to self.

fn put_u16(&mut self, n: u16)

Writes an unsigned 16 bit integer to self in big-endian byte order.

fn put_u16_le(&mut self, n: u16)

Writes an unsigned 16 bit integer to self in little-endian byte order.

fn put_i16(&mut self, n: i16)

Writes a signed 16 bit integer to self in big-endian byte order.

fn put_i16_le(&mut self, n: i16)

Writes a signed 16 bit integer to self in little-endian byte order.

fn put_u32(&mut self, n: u32)

Writes an unsigned 32 bit integer to self in big-endian byte order.

fn put_u32_le(&mut self, n: u32)

Writes an unsigned 32 bit integer to self in little-endian byte order.

fn put_i32(&mut self, n: i32)

Writes a signed 32 bit integer to self in big-endian byte order.

fn put_i32_le(&mut self, n: i32)

Writes a signed 32 bit integer to self in little-endian byte order.

fn put_u64(&mut self, n: u64)

Writes an unsigned 64 bit integer to self in the big-endian byte order.

fn put_u64_le(&mut self, n: u64)

Writes an unsigned 64 bit integer to self in little-endian byte order.

fn put_i64(&mut self, n: i64)

Writes a signed 64 bit integer to self in the big-endian byte order.

fn put_i64_le(&mut self, n: i64)

Writes a signed 64 bit integer to self in little-endian byte order.

fn put_u128(&mut self, n: u128)

Writes an unsigned 128 bit integer to self in the big-endian byte order.

fn put_u128_le(&mut self, n: u128)

Writes an unsigned 128 bit integer to self in little-endian byte order.

fn put_i128(&mut self, n: i128)

Writes a signed 128 bit integer to self in the big-endian byte order.

fn put_i128_le(&mut self, n: i128)

Writes a signed 128 bit integer to self in little-endian byte order.

fn put_uint(&mut self, n: u64, nbytes: usize)

Writes an unsigned n-byte integer to self in big-endian byte order.

fn put_uint_le(&mut self, n: u64, nbytes: usize)

Writes an unsigned n-byte integer to self in the little-endian byte order.

fn put_int(&mut self, n: i64, nbytes: usize)

Writes a signed n-byte integer to self in big-endian byte order.

fn put_int_le(&mut self, n: i64, nbytes: usize)

Writes a signed n-byte integer to self in little-endian byte order.

fn put_f32(&mut self, n: f32)

Writes an IEEE754 single-precision (4 bytes) floating point number to self in big-endian byte order.

fn put_f32_le(&mut self, n: f32)

Writes an IEEE754 single-precision (4 bytes) floating point number to self in little-endian byte order.

fn put_f64(&mut self, n: f64)

Writes an IEEE754 double-precision (8 bytes) floating point number to self in big-endian byte order.

fn put_f64_le(&mut self, n: f64)

Writes an IEEE754 double-precision (8 bytes) floating point number to self in little-endian byte order.

fn limit(self, limit: usize) -> Limit<Self>

where Self: Sized,

Creates an adaptor which can write at most limit bytes to self.

fn writer(self) -> Writer<Self>

where Self: Sized,

Creates an adaptor which implements the Write trait for self.

fn chain_mut<U>(self, next: U) -> Chain<Self, U>

where U: BufMut, Self: Sized,

Creates an adapter which will chain this buffer with another.

impl Debug for ChunkedBytes

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for ChunkedBytes

fn default() -> ChunkedBytes

Returns the “default value” for a type.

impl Write for ChunkedBytes

fn write_str(&mut self, s: &str) -> Result

Writes a string slice into this writer, returning whether the write succeeded.

fn write_fmt(&mut self, args: Arguments<'_>) -> Result

Glue for usage of the write! macro with implementors of this trait.

fn write_char(&mut self, c: char) -> Result<(), Error>

Writes a char into this writer, returning whether the write succeeded.