Struct fixed_buffer::FixedBuf

pub struct FixedBuf<T> { /* fields omitted */ }

FixedBuf is a fixed-length byte buffer. You can write bytes to it and then read them back.

It is not a circular buffer. Call shift periodically to move unread bytes to the front of the buffer.

Use new to make structs:

• FixedBuf<[u8; N]>
• FixedBuf<Box<[u8]>>
• FixedBuf<&mut [u8]>

FixedBuf<Box<[u8]>> uses less memory than Box<FixedBuf<[u8; N]>>. See new for details.

Implementations

impl<T> FixedBuf<T>

pub const fn new(mem: T) -> Self

Makes a new empty buffer, consuming or borrowing mem and using it as the internal memory array.

Creates FixedBuf<[u8; N]>, FixedBuf<Box<[u8]>>, and FixedBuf<&mut [u8]> structs.

This function is the inverse of into_inner.

FixedBuf<&mut [u8]> uses borrowed memory. Create one like this:

let mut mem = [0u8; 42];
let mut buf: FixedBuf<&mut [u8]> = FixedBuf::new(&mut mem);

FixedBuf<[u8; N]> can live on the stack. Be careful of stack overflows! Create one like this:

let mut buf: FixedBuf<[u8; 42]> = FixedBuf::new([0u8; 42]);

FixedBuf<Box<[u8]>> stores its memory block on the heap. Box supports coercion of Box<[T; N]> to Box<[T]>. Use that to create a boxed buffer:

let mut buf: FixedBuf<Box<[u8]>> = FixedBuf::new(Box::new([0u8; 42]));
// Your editor may incorrectly report "mismatched types [E0308]".

Note that FixedBuf<Box<[u8]>> is 10-25% more memory efficient than Box<FixedBuf<[u8; N]>>. Explanation:

Standard heaps allocate memory in blocks. The block sizes are powers of two and some intervening sizes. For example, jemalloc's block sizes are 8, 16, 32, 48, 64, 80, 96, 112, 128, 160, 192, 224, 256 bytes, and so on. Thus jemalloc uses 160 bytes to store a 129 byte value.

Every FixedBuf<[u8; N]> contains two usize index values in addition to its buffer memory. If you create a buffer with a power-of-two size, the struct is always a few bytes larger than a power of two, and takes up the next larger block size on the heap. For example, in a 64-bit program using jemalloc, Box<FixedBuf<[u8; 128]>> uses 128 + 8 + 8 = 144 bytes, and gets stored in a 160 byte block, wasting an extra 11% of memory.

By comparison, FixedBuf<Box<[u8]>> keeps the buffer memory separate from the index values and therefore wastes no memory. This is because Box<[u8; 128]> uses exactly 128-bytes on the heap.

Run the program box_benchmark/box_benchmark.rs to see the memory usage difference.

pub fn into_inner(self) -> T

Drops the struct and returns its internal array.

This function is the inverse of new.

pub fn len(&self) -> usize

Returns the number of unread bytes in the buffer.

Example:

let mut buf: FixedBuf<[u8; 16]> = FixedBuf::new([0u8; 16]);
assert_eq!(0, buf.len());
buf.write_str("abc");
assert_eq!(3, buf.len());
buf.read_bytes(2);
assert_eq!(1, buf.len());
buf.shift();
assert_eq!(1, buf.len());
buf.read_all();
assert_eq!(0, buf.len());

pub fn is_empty(&self) -> bool

Returns true if there are unread bytes in the buffer.

Example:

let mut buf: FixedBuf<[u8; 16]> = FixedBuf::new([0u8; 16]);
assert!(buf.is_empty());
buf.write_str("abc").unwrap();
assert!(!buf.is_empty());
buf.read_all();
assert!(buf.is_empty());

pub fn clear(&mut self)

Discards all data in the buffer.

impl<T: AsRef<[u8]>> FixedBuf<T>

pub fn filled(mem: T) -> Self

Makes a new full buffer, consuming or borrowing mem and using it as the internal memory array. Reading the buffer will return the bytes in mem.

You can write to the returned buf if mem implements AsMut<[u8].

For details, see new.

Examples:

// Readable, not writable:
let mut buf1 = FixedBuf::filled(b"abc");

// Readable and writable:
let mut buf2 = FixedBuf::filled([0u8; 42]);
let mut buf3: FixedBuf<[u8; 42]> = FixedBuf::filled([0u8; 42]);
let mut buf4: FixedBuf<Box<[u8]>> = FixedBuf::filled(Box::new([0u8; 42]));
// Your editor may incorrectly report "mismatched types [E0308]" --^

pub fn capacity(&self) -> usize

Returns the maximum number of bytes that can be stored in the buffer.

Example:

let mut buf: FixedBuf<[u8; 16]> = FixedBuf::new([0u8; 16]);
assert_eq!(16, buf.capacity());
buf.write_str("abc").unwrap();
assert_eq!(16, buf.capacity());

pub fn escape_ascii(&self) -> String

Copies all readable bytes to a string. Includes printable ASCII characters as-is. Converts non-printable characters to strings like "\n" and "\x19".

Leaves the buffer unchanged.

Uses core::ascii::escape_default(https://doc.rust-lang.org/core/ascii/fn.escape_default.html internally to escape each byte.

This function is useful for printing byte slices to logs and comparing byte slices in tests.

Example test:

use fixed_buffer::FixedBuf;
let mut buf = FixedBuf::new([0u8; 16]);
buf.write_str("abc");
buf.write_str("€");
assert_eq!("abc\\xe2\\x82\\xac", buf.escape_ascii());

pub fn mem(&self) -> &[u8]

This is a low-level function.

Borrows the entire internal memory buffer.

pub fn readable(&self) -> &[u8]

Returns the slice of readable bytes in the buffer. After processing some bytes from the front of the slice, call read to consume the bytes.

This is a low-level method. You probably want to use std::io::Read::read or tokio::io::AsyncReadExt::read , implemented for FixedBuffer in fixed_buffer_tokio::AsyncReadExt.

pub fn read_bytes(&mut self, num_bytes: usize) -> &[u8]

Reads bytes from the buffer.

Panics if the buffer does not contain enough bytes.

pub fn read_all(&mut self) -> &[u8]

Reads all the bytes from the buffer.

The buffer becomes empty and subsequent writes can fill the whole buffer.

pub fn read_and_copy_bytes(&mut self, dest: &mut [u8]) -> usize

Reads byte from the buffer and copies them into dest.

Returns the number of bytes copied.

Returns 0 when the buffer is empty or dest is zero-length.

impl<T: AsMut<[u8]>> FixedBuf<T>

pub fn copy_once_from<R: Read>(
    &mut self,
    reader: &mut R
) -> Result<usize, Error>

Reads from reader once and writes the data into the buffer.

Returns InvalidData if there is no empty space in the buffer. See shift.

pub fn write_str(&mut self, s: &str) -> Result<(), NotEnoughSpaceError>

Writes s into the buffer, after any unread bytes.

Returns Err if the buffer doesn't have enough free space at the end for the whole string.

See shift.

Example:

let mut buf: FixedBuf<[u8; 8]> = FixedBuf::new([0u8; 8]);
buf.write_str("123").unwrap();
buf.write_str("456").unwrap();
assert_eq!("1234", escape_ascii(buf.read_bytes(4)));
buf.write_str("78").unwrap();
buf.write_str("9").unwrap_err();  // End of buffer is full.

pub fn write_bytes(&mut self, data: &[u8]) -> Result<usize, NotEnoughSpaceError>

Tries to write data into the buffer, after any unread bytes.

Returns Ok(data.len()) if it wrote all of the bytes.

Returns NotEnoughSpaceError if the buffer doesn't have enough free space at the end for all of the bytes.

See shift.

Example:

let mut buf: FixedBuf<[u8; 8]> = FixedBuf::new([0u8; 8]);
assert_eq!(3 as usize, buf.write_bytes("123".as_bytes()).unwrap());
assert_eq!(3 as usize, buf.write_bytes("456".as_bytes()).unwrap());
assert_eq!("1234", escape_ascii(buf.read_bytes(4)));
assert_eq!(2 as usize, buf.write_bytes("78".as_bytes()).unwrap());  // Fills buffer.
buf.write_bytes("9".as_bytes()).unwrap_err();  // Error, buffer is full.

pub fn writable(&mut self) -> Option<&mut [u8]>

Returns the writable part of the buffer.

To use this, first modify bytes at the beginning of the slice. Then call wrote(usize) to commit those bytes into the buffer and make them available for reading.

Returns None when the end of the buffer is full. See shift.

This is a low-level method. You probably want to use std::io::Write::write or tokio::io::AsyncWriteExt::write, implemented for FixedBuffer in fixed_buffer_tokio::AsyncWriteExt.

Example:

let mut buf: FixedBuf<[u8; 8]> = FixedBuf::new([0u8; 8]);
buf.writable().unwrap()[0] = 'a' as u8;
buf.writable().unwrap()[1] = 'b' as u8;
buf.writable().unwrap()[2] = 'c' as u8;
buf.wrote(3);
assert_eq!("abc", escape_ascii(buf.read_bytes(3)));

pub fn wrote(&mut self, num_bytes: usize)

Commits bytes into the buffer. Call this after writing to the front of the writable slice.

This is a low-level method.

Panics when there is not num_bytes free at the end of the buffer.

See writable().

pub fn shift(&mut self)

Recovers buffer space.

The buffer is not circular. After you read bytes, the space at the beginning of the buffer is unused. Call this method to move unread data to the beginning of the buffer and recover the space. This makes the free space available for writes, which go at the end of the buffer.

impl<T: AsRef<[u8]> + AsMut<[u8]>> FixedBuf<T>

pub fn deframe<F>(
    &mut self,
    deframer_fn: F
) -> Result<Option<Range<usize>>, Error> where
    F: Fn(&[u8]) -> Result<Option<(Range<usize>, usize)>, MalformedInputError>, 

This is a low-level function. Use read_frame instead.

Calls deframer_fn to check if the buffer contains a complete frame. Consumes the frame bytes from the buffer and returns the range of the frame's contents in the internal memory. Use mem to immutably borrow the internal memory and construct the slice with &mem()[range]. This is necessary because deframe borrows self mutably but read_frame needs to borrow it immutably and return a slice.

Returns None if the buffer is empty or contains an incomplete frame.

Returns InvalidData when deframer_fn returns an error.

pub fn read_frame<R, F>(
    &mut self,
    reader: &mut R,
    deframer_fn: F
) -> Result<Option<&[u8]>, Error> where
    R: Read,
    F: Fn(&[u8]) -> Result<Option<(Range<usize>, usize)>, MalformedInputError>, 

Reads from reader into the buffer.

After each read, calls deframer_fn to check if the buffer now contains a complete frame. Consumes the frame bytes from the buffer and returns a slice with the frame contents.

Returns None when reader reaches EOF and the buffer is empty.

Returns UnexpectedEof when reader reaches EOF and the buffer contains an incomplete frame.

Returns InvalidData when deframer_fn returns an error or the buffer fills up.

Calls shift before reading.

Provided deframer functions:

• deframe_line
• deframe_crlf

Example

let mut buf: FixedBuf<[u8; 32]> = FixedBuf::new([0u8; 32]);
let mut input = std::io::Cursor::new(b"aaa\r\nbbb\n\nccc\n");
assert_eq!("aaa", escape_ascii(buf.read_frame(&mut input, deframe_line).unwrap().unwrap()));
assert_eq!("bbb", escape_ascii(buf.read_frame(&mut input, deframe_line).unwrap().unwrap()));
assert_eq!("", escape_ascii(buf.read_frame(&mut input, deframe_line).unwrap().unwrap()));
assert_eq!("ccc", escape_ascii(buf.read_frame(&mut input, deframe_line).unwrap().unwrap()));
assert_eq!(None, buf.read_frame(&mut input, deframe_line).unwrap());

Deframer Function deframe_fn

Checks if data contains an entire frame.

Never panics.

Returns Err(MalformedInputError) if data contains a malformed frame.

Returns Ok(None) if data contains an incomplete frame. The caller will read more data and call deframe again.

Returns Ok((range, len)) when data contains a complete well-formed frame of length len and contents &data[range].

A frame is a sequence of bytes containing a payload and metadata prefix or suffix bytes. The metadata define where the payload starts and ends. Popular frame protocols include line-delimiting (CSV), hexadecimal length prefix (HTTP chunked transfer encoding), and binary length prefix (TLS).

If the caller removes len bytes from the front of data, then data should start with the next frame, ready to call deframe again.

A line-delimited deframer returns range so that &data[range] contains the entire line without the final b'\n' or b"\r\n". It returns len that counts the bytes of the entire line and the final b'\n' or b"\r\n".

Example

A CRLF-terminated line deframer:

assert_eq!(Ok(None), deframe_crlf(b""));
assert_eq!(Ok(None), deframe_crlf(b"abc"));
assert_eq!(Ok(None), deframe_crlf(b"abc\r"));
assert_eq!(Ok(None), deframe_crlf(b"abc\n"));
assert_eq!(Ok(Some((0..3, 5))), deframe_crlf(b"abc\r\n"));
assert_eq!(Ok(Some((0..3, 5))), deframe_crlf(b"abc\r\nX"));

Trait Implementations

impl<T: Clone> Clone for FixedBuf<T>

pub fn clone(&self) -> FixedBuf<T>

Returns a copy of the value.

pub fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<T: Copy> Copy for FixedBuf<T>

impl<T: AsRef<[u8]>> Debug for FixedBuf<T>

pub fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl<T: Default> Default for FixedBuf<T>

pub fn default() -> FixedBuf<T>

Returns the "default value" for a type.

impl<T: Eq> Eq for FixedBuf<T>

impl<T: Hash> Hash for FixedBuf<T>

pub fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

pub fn hash_slice<H>(data: &[Self], state: &mut H) where
    H: Hasher, 

Feeds a slice of this type into the given Hasher.

impl<T: PartialEq> PartialEq<FixedBuf<T>> for FixedBuf<T>

pub fn eq(&self, other: &FixedBuf<T>) -> bool

This method tests for self and other values to be equal, and is used by ==.

pub fn ne(&self, other: &FixedBuf<T>) -> bool

This method tests for !=.

impl<T: AsRef<[u8]>> Read for FixedBuf<T>

pub fn read(&mut self, buf: &mut [u8]) -> Result<usize>

Pull some bytes from this source into the specified buffer, returning how many bytes were read.

pub fn read_vectored(
    &mut self,
    bufs: &mut [IoSliceMut<'_>]
) -> Result<usize, Error>

Like read, except that it reads into a slice of buffers.

pub fn is_read_vectored(&self) -> bool

🔬 This is a nightly-only experimental API. (can_vector)

Determines if this Reader has an efficient read_vectored implementation.

pub unsafe fn initializer(&self) -> Initializer

🔬 This is a nightly-only experimental API. (read_initializer)

Determines if this Reader can work with buffers of uninitialized memory.

pub fn read_to_end(&mut self, buf: &mut Vec<u8, Global>) -> Result<usize, Error>

Read all bytes until EOF in this source, placing them into buf.

pub fn read_to_string(&mut self, buf: &mut String) -> Result<usize, Error>

Read all bytes until EOF in this source, appending them to buf.

pub fn read_exact(&mut self, buf: &mut [u8]) -> Result<(), Error>

Read the exact number of bytes required to fill buf.

pub fn by_ref(&mut self) -> &mut Self

Creates a "by reference" adaptor for this instance of Read.

pub fn bytes(self) -> Bytes<Self>

Transforms this Read instance to an Iterator over its bytes.

pub fn chain<R>(self, next: R) -> Chain<Self, R> where
    R: Read, 

Creates an adaptor which will chain this stream with another.

pub fn take(self, limit: u64) -> Take<Self>

Creates an adaptor which will read at most limit bytes from it.

impl<T> StructuralEq for FixedBuf<T>

impl<T> StructuralPartialEq for FixedBuf<T>

impl<T: AsMut<[u8]>> Write for FixedBuf<T>

pub fn write(&mut self, data: &[u8]) -> Result<usize>

Write a buffer into this writer, returning how many bytes were written.

pub fn flush(&mut self) -> Result<()>

Flush this output stream, ensuring that all intermediately buffered contents reach their destination.

pub fn write_vectored(&mut self, bufs: &[IoSlice<'_>]) -> Result<usize, Error>

Like write, except that it writes from a slice of buffers.

pub fn is_write_vectored(&self) -> bool

🔬 This is a nightly-only experimental API. (can_vector)

Determines if this Writer has an efficient write_vectored implementation.

pub fn write_all(&mut self, buf: &[u8]) -> Result<(), Error>

Attempts to write an entire buffer into this writer.

pub fn write_all_vectored(
    &mut self,
    bufs: &mut [IoSlice<'_>]
) -> Result<(), Error>

🔬 This is a nightly-only experimental API. (write_all_vectored)

Attempts to write multiple buffers into this writer.

pub fn write_fmt(&mut self, fmt: Arguments<'_>) -> Result<(), Error>

Writes a formatted string into this writer, returning any error encountered.

pub fn by_ref(&mut self) -> &mut Self

Creates a "by reference" adaptor for this instance of Write.