[−][src]Struct fixed_buffer::FixedBuf
FixedBuf is a fixed-length byte buffer. You can write bytes to it and then read them back.
It implements tokio's AsyncRead
and AsyncWrite
traits.
Use read_delimited
to read lines and other delimited messages.
This works like tokio::io::AsyncBufReadExt::read_until
,
but uses a fixed sized buffer so network peers cannot OOM the process.
It is not a circular buffer. Call shift
periodically to
move unread bytes to the front of the buffer.
Use new
to create FixedBuf<[u8; N]>
and FixedBuf<Box<[u8; N]>>
structs
with sizes
8, 16, 32, 64, 100, 128, 200, 256, 512 bytes,
1KB, 2KB, 4KB, 8KB, 16KB, 32KB 64KB, 128KB, 512KB, and 1MB.
Note that FixedBuf<Box<[u8; N]>>
uses less memory than Box<FixedBuf<[u8; N]>>
.
See new
for details.
Use new_with_slice
to create a FixedBuf<&mut [u8]>
that uses a borrowed slice of any size.
Implementations
impl<'_> FixedBuf<&'_ mut [u8]>
[src]
pub fn new_with_slice(mem: &mut [u8]) -> FixedBuf<&mut [u8]>ⓘ
[src]
Makes a new buffer with the provided slice as the internal memory.
impl<T: OwnedMemBlock> FixedBuf<T>
[src]
pub fn new() -> Self
[src]
Makes a new buffer with an internal memory array.
Creates FixedBuf<[u8; N]>
and FixedBuf<Box<[u8; N]>>
structs
with sizes
8, 16, 32, 64, 100, 128, 200, 256, 512 bytes,
1KB, 2KB, 4KB, 8KB, 16KB, 32KB 64KB, 128KB, 512KB, and 1MB.
Use [new_with_slice
] to create a FixedBuf<&mut [u8]>
that uses a borrowed slice of any size.
FixedBuf<[u8; N]>
is stored on the stack. Be careful of stack overflows!
FixedBuf<Box<[u8; N]>>
stores its memory block on the heap.
Note that FixedBuf<Box<[u8; N]>>
is 15% more memory efficient than
Box<FixedBuf<[u8; N]>>
. Explanation:
Standard heaps allocate memory in blocks. The block sizes are powers of two and some intervening sizes. For example, jemalloc's block sizes are 8, 16, 32, 48, 64, 80, 96, 112, 128, 160, 192, 224, 256 bytes, and so on. Thus jemalloc uses 160 bytes to store a 129 byte value.
Every FixedBuf<[u8; N]>
contains two usize
index values
in addition to its buffer memory.
Since the supported buffer sizes are powers of two, the struct size is always a few
bytes larger than a power of two, and takes up the next larger block size on the
heap. For example, in a 64-bit program using jemalloc,
Box<FixedBuf<[u8; 128]>
> uses 128 + 8 + 8 (buffer + read_index + write_index) = 144 bytes,
and gets stored in a 160 byte block, wasting an extra 11% of memory.
Use FixedBuf<Box<[u8; N]>>
to avoid wasting memory.
Those structs allocate their buffer memory separately.
A FixedBuf<Box<[u8; 128]>>
allocates a 128-byte array on the heap and wastes no memory.
Run the program [examples/box_benchmark.rs
] to see the memory usage
difference.
[examples/box_benchmark.rs
]: examples/box_benchmark.rs
[new_with_slice
]: #method.new_with_slice
pub fn new_with_mem(mem: T) -> Self
[src]
Consumes the provided array and uses it in a new empty buffer.
This function is the inverse of into_inner
.
pub fn into_inner(self) -> T
[src]
Drops the struct and returns its internal array.
This function is the inverse of new_with_mem
.
impl<T: MemBlock> FixedBuf<T>
[src]
pub fn len(&self) -> usize
[src]
Returns the number of unread bytes in the buffer.
Example:
let mut buf: FixedBuf<[u8; 16]> = FixedBuf::new(); assert_eq!(0, buf.len()); buf.write_str("abc"); assert_eq!(3, buf.len()); buf.read_bytes(2); assert_eq!(1, buf.len()); buf.shift(); assert_eq!(1, buf.len()); buf.read_all(); assert_eq!(0, buf.len());
pub fn capacity(&self) -> usize
[src]
Returns the maximum number of bytes that can be stored in the buffer.
Example:
let mut buf: FixedBuf<[u8; 16]> = FixedBuf::new(); assert_eq!(16, buf.capacity()); buf.write_str("abc").unwrap(); assert_eq!(16, buf.capacity());
pub fn is_empty(&self) -> bool
[src]
Returns true if there are unread bytes in the buffer.
Example:
let mut buf: FixedBuf<[u8; 16]> = FixedBuf::new(); assert!(buf.is_empty()); buf.write_str("abc").unwrap(); assert!(!buf.is_empty()); buf.read_all(); assert!(buf.is_empty());
pub fn write_str(&mut self, s: &str) -> Result<()>
[src]
Writes s
into the buffer, after any unread bytes.
Returns Err
if the buffer doesn't have enough free space at the end
for the whole string.
See shift
.
Example:
let mut buf: FixedBuf<[u8; 8]> = FixedBuf::new(); buf.write_str("123").unwrap(); buf.write_str("456").unwrap(); assert_eq!("1234", escape_ascii(buf.read_bytes(4))); buf.write_str("78").unwrap(); buf.write_str("9").unwrap_err(); // End of buffer is full.
pub fn write_bytes(&mut self, data: &[u8]) -> Result<usize>
[src]
Try to write data
into the buffer, after any unread bytes.
Returns Ok(data.len())
if it wrote all of the bytes.
Returns Err
if the buffer doesn't have enough free space at the end
for all of the bytes.
See shift
.
Example:
let mut buf: FixedBuf<[u8; 8]> = FixedBuf::new(); assert_eq!(3 as usize, buf.write_bytes("123".as_bytes()).unwrap()); assert_eq!(3 as usize, buf.write_bytes("456".as_bytes()).unwrap()); assert_eq!("1234", escape_ascii(buf.read_bytes(4))); assert_eq!(2 as usize, buf.write_bytes("78".as_bytes()).unwrap()); // Fills buffer. buf.write_bytes("9".as_bytes()).unwrap_err(); // Error, buffer is full.
pub fn writable(&mut self) -> Option<&mut [u8]>
[src]
Returns the writable part of the buffer.
To use this, first modify bytes at the beginning of the slice.
Then call wrote(usize)
to commit those bytes into the buffer
and make them available for reading.
Returns None
when the end of the buffer is full. See shift
.
This is a low-level method.
You probably want to use std::io::Write::write
and tokio::io::AsyncWriteExt::write
instead.
Example:
let mut buf: FixedBuf<[u8; 8]> = FixedBuf::new(); buf.writable().unwrap()[0] = 'a' as u8; buf.writable().unwrap()[1] = 'b' as u8; buf.writable().unwrap()[2] = 'c' as u8; buf.wrote(3); assert_eq!("abc", escape_ascii(buf.read_bytes(3)));
pub fn wrote(&mut self, num_bytes: usize)
[src]
Commit bytes into the buffer.
Call this after writing to the front of the writable
slice.
This is a low-level method.
Panics if writable()
is not large enough.
See example in writable()
.
pub fn readable(&self) -> &[u8]
[src]
Returns the slice of readable bytes in the buffer.
After processing some bytes from the front of the slice, call read
to consume the bytes.
This is a low-level method.
You probably want to use
read
,
std::io::Read::read
,
and tokio::io::AsyncReadExt::read
instead.
Example:
fn try_process_record(b: &[u8]) -> Result<usize, Error> { if b.len() < 2 { return Ok(0) } if b.starts_with("ab".as_bytes()) { println!("found record"); Ok(2) } else { Err(Error::new(ErrorKind::InvalidData, "bad record")) } } async fn read_and_process<R: tokio::io::AsyncRead + Unpin>(mut input: R) -> Result<(), Error> { let mut buf: FixedBuf<[u8; 1024]> = FixedBuf::new(); loop { // Read a chunk into the buffer. let mut writable = buf.writable() .ok_or(Error::new(ErrorKind::InvalidData, "record too long, buffer full"))?; let bytes_written = AsyncReadExt::read(&mut input, &mut writable).await?; if bytes_written == 0 { return if buf.len() == 0 { Ok(()) // EOF at record boundary } else { // EOF in the middle of a record Err(Error::from(ErrorKind::UnexpectedEof)) }; } buf.wrote(bytes_written); // Process records in the buffer. loop { let bytes_read = try_process_record(buf.readable())?; if bytes_read == 0 { break; } buf.read_bytes(bytes_read); } // Shift data in the buffer to free up space at the end for writing. buf.shift(); } } read_and_process(std::io::Cursor::new(b"")).await.unwrap(); read_and_process(std::io::Cursor::new(b"abab")).await.unwrap(); assert_eq!( std::io::ErrorKind::UnexpectedEof, read_and_process(std::io::Cursor::new(b"aba")).await.unwrap_err().kind() );
pub fn read_bytes(&mut self, num_bytes: usize) -> &[u8]
[src]
Read bytes from the buffer.
Panics if the buffer does not contain enough bytes.
pub fn read_all(&mut self) -> &[u8]
[src]
Read all the bytes from the buffer. The buffer becomes empty and subsequent writes can fill the whole buffer.
pub fn read_and_copy_bytes(&mut self, dest: &mut [u8]) -> Result<usize>
[src]
Reads bytes from the buffer and copies them into dest
.
Returns the number of bytes copied.
Returns Ok(0)
when the buffer is empty or dest
is zero-length.
pub async fn read_delimited<R, '_, '_, '_>(
&'_ mut self,
__arg1: R,
delim: &'_ [u8]
) -> Result<&'_ [u8]> where
R: AsyncRead + Unpin + Send,
[src]
&'_ mut self,
__arg1: R,
delim: &'_ [u8]
) -> Result<&'_ [u8]> where
R: AsyncRead + Unpin + Send,
Reads from a tokio::io::AsyncRead
into the buffer until it finds delim
.
Returns the slice up until delim
.
Consumes the returned bytes and delim
.
Leaves unused bytes in the buffer.
If the buffer already contains delim
,
returns the data immediately without reading from input
.
If the buffer does not already contain delim
, calls shift
before
reading from input
.
Returns Err(Error(InvalidData,_))
if the buffer fills up before delim
is found.
Demo:
let mut buf: FixedBuf<[u8; 32]> = FixedBuf::new(); let mut input = std::io::Cursor::new(b"aaa\nbbb\n\nccc\n"); assert_eq!("aaa", escape_ascii(buf.read_delimited(&mut input, b"\n").await.unwrap())); assert_eq!("bbb", escape_ascii(buf.read_delimited(&mut input, b"\n").await.unwrap())); assert_eq!("", escape_ascii(buf.read_delimited(&mut input, b"\n").await.unwrap())); assert_eq!("ccc", escape_ascii(buf.read_delimited(&mut input, b"\n").await.unwrap())); assert_eq!( std::io::ErrorKind::NotFound, buf.read_delimited(&mut input, b"\n").await.unwrap_err().kind() );
Example usage:
let (mut input, mut output) = tcp_stream.split(); let mut buf: FixedBuf<[u8; 1024]> = FixedBuf::new(); loop { // Read a line and leave leftover bytes in `buf`. let line_bytes: &[u8] = buf.read_delimited(&mut input, b"\n").await?; let request = Request::parse(line_bytes)?; // Read any request payload from `buf` + `TcpStream`. let payload_reader = tokio::io::AsyncReadExt::chain(&mut buf, &mut input); handle_request(&mut output, payload_reader, request).await?; }
pub fn shift(&mut self)
[src]
Recovers buffer space.
The buffer is not circular. After you read bytes, the space at the beginning of the buffer is unused. Call this method to move unread data to the beginning of the buffer and recover the space. This makes the free space available for writes, which go at the end of the buffer.
For an example, see readable
.
Trait Implementations
impl<T: MemBlock + Unpin> AsyncRead for FixedBuf<T>
[src]
fn poll_read(
self: Pin<&mut Self>,
_cx: &mut Context<'_>,
buf: &mut ReadBuf<'_>
) -> Poll<Result<(), Error>>
[src]
self: Pin<&mut Self>,
_cx: &mut Context<'_>,
buf: &mut ReadBuf<'_>
) -> Poll<Result<(), Error>>
impl<T: MemBlock + Unpin> AsyncWrite for FixedBuf<T>
[src]
fn poll_write(
self: Pin<&mut Self>,
_cx: &mut Context<'_>,
buf: &[u8]
) -> Poll<Result<usize, Error>>
[src]
self: Pin<&mut Self>,
_cx: &mut Context<'_>,
buf: &[u8]
) -> Poll<Result<usize, Error>>
fn poll_flush(
self: Pin<&mut Self>,
_cx: &mut Context<'_>
) -> Poll<Result<(), Error>>
[src]
self: Pin<&mut Self>,
_cx: &mut Context<'_>
) -> Poll<Result<(), Error>>
fn poll_shutdown(
self: Pin<&mut Self>,
_cx: &mut Context<'_>
) -> Poll<Result<(), Error>>
[src]
self: Pin<&mut Self>,
_cx: &mut Context<'_>
) -> Poll<Result<(), Error>>
impl<T: Clone + MemBlock> Clone for FixedBuf<T>
[src]
impl<T: MemBlock> Debug for FixedBuf<T>
[src]
impl<T: OwnedMemBlock> Default for FixedBuf<T>
[src]
impl<T: Eq + MemBlock> Eq for FixedBuf<T>
[src]
impl<T: Hash + MemBlock> Hash for FixedBuf<T>
[src]
fn hash<__H: Hasher>(&self, state: &mut __H)
[src]
fn hash_slice<H>(data: &[Self], state: &mut H) where
H: Hasher,
1.3.0[src]
H: Hasher,
impl<T: PartialEq + MemBlock> PartialEq<FixedBuf<T>> for FixedBuf<T>
[src]
impl<T: MemBlock> Read for FixedBuf<T>
[src]
fn read(&mut self, buf: &mut [u8]) -> Result<usize>
[src]
fn read_vectored(&mut self, bufs: &mut [IoSliceMut<'_>]) -> Result<usize, Error>
1.36.0[src]
fn is_read_vectored(&self) -> bool
[src]
unsafe fn initializer(&self) -> Initializer
[src]
fn read_to_end(&mut self, buf: &mut Vec<u8>) -> Result<usize, Error>
1.0.0[src]
fn read_to_string(&mut self, buf: &mut String) -> Result<usize, Error>
1.0.0[src]
fn read_exact(&mut self, buf: &mut [u8]) -> Result<(), Error>
1.6.0[src]
fn by_ref(&mut self) -> &mut Self
1.0.0[src]
fn bytes(self) -> Bytes<Self>
1.0.0[src]
fn chain<R>(self, next: R) -> Chain<Self, R> where
R: Read,
1.0.0[src]
R: Read,
fn take(self, limit: u64) -> Take<Self>
1.0.0[src]
impl<T: MemBlock> StructuralEq for FixedBuf<T>
[src]
impl<T: MemBlock> StructuralPartialEq for FixedBuf<T>
[src]
impl<T: MemBlock> Write for FixedBuf<T>
[src]
fn write(&mut self, data: &[u8]) -> Result<usize>
[src]
fn flush(&mut self) -> Result<()>
[src]
fn write_vectored(&mut self, bufs: &[IoSlice<'_>]) -> Result<usize, Error>
1.36.0[src]
fn is_write_vectored(&self) -> bool
[src]
fn write_all(&mut self, buf: &[u8]) -> Result<(), Error>
1.0.0[src]
fn write_all_vectored(&mut self, bufs: &mut [IoSlice<'_>]) -> Result<(), Error>
[src]
fn write_fmt(&mut self, fmt: Arguments<'_>) -> Result<(), Error>
1.0.0[src]
fn by_ref(&mut self) -> &mut Self
1.0.0[src]
Auto Trait Implementations
impl<T> RefUnwindSafe for FixedBuf<T> where
T: RefUnwindSafe,
T: RefUnwindSafe,
impl<T> Send for FixedBuf<T> where
T: Send,
T: Send,
impl<T> Sync for FixedBuf<T> where
T: Sync,
T: Sync,
impl<T> Unpin for FixedBuf<T> where
T: Unpin,
T: Unpin,
impl<T> UnwindSafe for FixedBuf<T> where
T: UnwindSafe,
T: UnwindSafe,
Blanket Implementations
impl<T> Any for T where
T: 'static + ?Sized,
[src]
T: 'static + ?Sized,
impl<R> AsyncReadExt for R where
R: AsyncRead + ?Sized,
[src]
R: AsyncRead + ?Sized,
fn chain<R>(self, next: R) -> Chain<Self, R> where
R: AsyncRead,
[src]
R: AsyncRead,
fn read(&'a mut self, buf: &'a mut [u8]) -> Read<'a, Self> where
Self: Unpin,
[src]
Self: Unpin,
fn read_exact(&'a mut self, buf: &'a mut [u8]) -> ReadExact<'a, Self> where
Self: Unpin,
[src]
Self: Unpin,
fn read_u8(&'a mut self) -> ReadU8<&'a mut Self> where
Self: Unpin,
[src]
Self: Unpin,
fn read_i8(&'a mut self) -> ReadI8<&'a mut Self> where
Self: Unpin,
[src]
Self: Unpin,
fn read_u16(&'a mut self) -> ReadU16<&'a mut Self> where
Self: Unpin,
[src]
Self: Unpin,
fn read_i16(&'a mut self) -> ReadI16<&'a mut Self> where
Self: Unpin,
[src]
Self: Unpin,
fn read_u32(&'a mut self) -> ReadU32<&'a mut Self> where
Self: Unpin,
[src]
Self: Unpin,
fn read_i32(&'a mut self) -> ReadI32<&'a mut Self> where
Self: Unpin,
[src]
Self: Unpin,
fn read_u64(&'a mut self) -> ReadU64<&'a mut Self> where
Self: Unpin,
[src]
Self: Unpin,
fn read_i64(&'a mut self) -> ReadI64<&'a mut Self> where
Self: Unpin,
[src]
Self: Unpin,
fn read_u128(&'a mut self) -> ReadU128<&'a mut Self> where
Self: Unpin,
[src]
Self: Unpin,
fn read_i128(&'a mut self) -> ReadI128<&'a mut Self> where
Self: Unpin,
[src]
Self: Unpin,
fn read_u16_le(&'a mut self) -> ReadU16Le<&'a mut Self> where
Self: Unpin,
[src]
Self: Unpin,
fn read_i16_le(&'a mut self) -> ReadI16Le<&'a mut Self> where
Self: Unpin,
[src]
Self: Unpin,
fn read_u32_le(&'a mut self) -> ReadU32Le<&'a mut Self> where
Self: Unpin,
[src]
Self: Unpin,
fn read_i32_le(&'a mut self) -> ReadI32Le<&'a mut Self> where
Self: Unpin,
[src]
Self: Unpin,
fn read_u64_le(&'a mut self) -> ReadU64Le<&'a mut Self> where
Self: Unpin,
[src]
Self: Unpin,
fn read_i64_le(&'a mut self) -> ReadI64Le<&'a mut Self> where
Self: Unpin,
[src]
Self: Unpin,
fn read_u128_le(&'a mut self) -> ReadU128Le<&'a mut Self> where
Self: Unpin,
[src]
Self: Unpin,
fn read_i128_le(&'a mut self) -> ReadI128Le<&'a mut Self> where
Self: Unpin,
[src]
Self: Unpin,
fn read_to_end(&'a mut self, buf: &'a mut Vec<u8>) -> ReadToEnd<'a, Self> where
Self: Unpin,
[src]
Self: Unpin,
fn read_to_string(&'a mut self, dst: &'a mut String) -> ReadToString<'a, Self> where
Self: Unpin,
[src]
Self: Unpin,
fn take(self, limit: u64) -> Take<Self>
[src]
impl<W> AsyncWriteExt for W where
W: AsyncWrite + ?Sized,
[src]
W: AsyncWrite + ?Sized,
fn write(&'a mut self, src: &'a [u8]) -> Write<'a, Self> where
Self: Unpin,
[src]
Self: Unpin,
fn write_all(&'a mut self, src: &'a [u8]) -> WriteAll<'a, Self> where
Self: Unpin,
[src]
Self: Unpin,
fn write_u8(&'a mut self, n: u8) -> WriteU8<&'a mut Self> where
Self: Unpin,
[src]
Self: Unpin,
fn write_i8(&'a mut self, n: i8) -> WriteI8<&'a mut Self> where
Self: Unpin,
[src]
Self: Unpin,
fn write_u16(&'a mut self, n: u16) -> WriteU16<&'a mut Self> where
Self: Unpin,
[src]
Self: Unpin,
fn write_i16(&'a mut self, n: i16) -> WriteI16<&'a mut Self> where
Self: Unpin,
[src]
Self: Unpin,
fn write_u32(&'a mut self, n: u32) -> WriteU32<&'a mut Self> where
Self: Unpin,
[src]
Self: Unpin,
fn write_i32(&'a mut self, n: i32) -> WriteI32<&'a mut Self> where
Self: Unpin,
[src]
Self: Unpin,
fn write_u64(&'a mut self, n: u64) -> WriteU64<&'a mut Self> where
Self: Unpin,
[src]
Self: Unpin,
fn write_i64(&'a mut self, n: i64) -> WriteI64<&'a mut Self> where
Self: Unpin,
[src]
Self: Unpin,
fn write_u128(&'a mut self, n: u128) -> WriteU128<&'a mut Self> where
Self: Unpin,
[src]
Self: Unpin,
fn write_i128(&'a mut self, n: i128) -> WriteI128<&'a mut Self> where
Self: Unpin,
[src]
Self: Unpin,
fn write_u16_le(&'a mut self, n: u16) -> WriteU16Le<&'a mut Self> where
Self: Unpin,
[src]
Self: Unpin,
fn write_i16_le(&'a mut self, n: i16) -> WriteI16Le<&'a mut Self> where
Self: Unpin,
[src]
Self: Unpin,
fn write_u32_le(&'a mut self, n: u32) -> WriteU32Le<&'a mut Self> where
Self: Unpin,
[src]
Self: Unpin,
fn write_i32_le(&'a mut self, n: i32) -> WriteI32Le<&'a mut Self> where
Self: Unpin,
[src]
Self: Unpin,
fn write_u64_le(&'a mut self, n: u64) -> WriteU64Le<&'a mut Self> where
Self: Unpin,
[src]
Self: Unpin,
fn write_i64_le(&'a mut self, n: i64) -> WriteI64Le<&'a mut Self> where
Self: Unpin,
[src]
Self: Unpin,
fn write_u128_le(&'a mut self, n: u128) -> WriteU128Le<&'a mut Self> where
Self: Unpin,
[src]
Self: Unpin,
fn write_i128_le(&'a mut self, n: i128) -> WriteI128Le<&'a mut Self> where
Self: Unpin,
[src]
Self: Unpin,
fn flush(&mut self) -> Flush<'_, Self> where
Self: Unpin,
[src]
Self: Unpin,
fn shutdown(&mut self) -> Shutdown<'_, Self> where
Self: Unpin,
[src]
Self: Unpin,
impl<T> Borrow<T> for T where
T: ?Sized,
[src]
T: ?Sized,
impl<T> BorrowMut<T> for T where
T: ?Sized,
[src]
T: ?Sized,
pub fn borrow_mut(&mut self) -> &mut T
[src]
impl<T> From<T> for T
[src]
impl<T, U> Into<U> for T where
U: From<T>,
[src]
U: From<T>,
impl<T> ToOwned for T where
T: Clone,
[src]
T: Clone,
type Owned = T
The resulting type after obtaining ownership.
pub fn to_owned(&self) -> T
[src]
pub fn clone_into(&self, target: &mut T)
[src]
impl<T, U> TryFrom<U> for T where
U: Into<T>,
[src]
U: Into<T>,
type Error = Infallible
The type returned in the event of a conversion error.
pub fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>
[src]
impl<T, U> TryInto<U> for T where
U: TryFrom<T>,
[src]
U: TryFrom<T>,