fixed-buffer 0.1.6

Fixed-size buffers for network protocol parsers
Documentation

fixed-buffer

This is a Rust library with fixed-size buffers, useful for network protocol parsers.

Features

  • Write bytes to the buffer and read them back
  • Lives on the stack
  • Does not allocate memory
  • Supports tokio's AsyncRead and AsyncWrite
  • Use it to read a stream, search for a delimiter, and save leftover bytes for the next read.
  • Easy to learn & use. Easy to maintain code that uses it.
  • Works with Rust latest, beta, and nightly
  • No macros

Limitations

  • Not a circular buffer. You can call shift() periodically to move unread bytes to the front of the buffer.
  • There is no iterate_delimited(AsyncRead). Because of borrowing rules, such a function would need to return non-borrowed (allocated and copied) data.

Examples

Read and handle requests from a remote client:

use fixed_buffer::FixedBuf;
use std::io::Error;
use tokio::io::AsyncWriteExt;
use tokio::net::TcpStream;

async fn handle_conn(mut tcp_stream: TcpStream) -> Result<(), Error> {
    let (mut input, mut output) = tcp_stream.split();
    let mut buf: FixedBuf<[u8; 4096]> = FixedBuf::new([0; 4096]);
    loop {
        // Read a line and leave leftover bytes in `buf`.
        let line_bytes = match buf.read_delimited(&mut input, b"\n").await? {
            Some(line_bytes) => line_bytes,
            None => return Ok(()),
        };
        let request = Request::parse(line_bytes)?;
        // Read any request payload from `buf` + `TcpStream`.
        let payload_reader = tokio::io::AsyncReadExt::chain(&mut buf, &mut input);
        handle_request(&mut output, payload_reader, request).await?;
    }
}

For a runnable example, see examples/server.rs.

Read and process records:

fn try_process_record(b: &[u8]) -> Result<usize, Error> {
    if b.len() < 2 {
        return Ok(0)
    }
    if b.starts_with("ab".as_bytes()) {
        println!("found record");
        Ok(2)
    } else {
        Err(Error::new(ErrorKind::InvalidData, "bad record"))
    }
}

async fn read_and_process<R: tokio::io::AsyncRead + Unpin>(mut input: R)
    -> Result<(), Error> {
    let mut buf: FixedBuf<[u8; 1024]> = FixedBuf::new([0; 1024]);
    loop {
        // Read a chunk into the buffer.
        let mut writable = buf.writable()
            .ok_or(Error::new(ErrorKind::InvalidData, "record too long, buffer full"))?;
        let bytes_written = AsyncReadExt::read(&mut input, &mut writable).await?;
        if bytes_written == 0 {
            return if buf.len() == 0 {
                Ok(())  // EOF at record boundary
            } else {
                // EOF in the middle of a record
                Err(Error::from(ErrorKind::UnexpectedEof))
            };
        }
        buf.wrote(bytes_written);

        // Process records in the buffer.
        loop {
            let bytes_read = try_process_record(buf.readable())?;
            if bytes_read == 0 {
                break;
            }
            buf.read_bytes(bytes_read);
        }
        // Shift data in the buffer to free up space at the end for writing.
        buf.shift();
    }
}

Documentation

https://docs.rs/fixed-buffer

The filled constructor is useful in tests.

Alternatives

Release Process

  1. Edit Cargo.toml and bump version number.
  2. Run ./release.sh

Changelog

  • v0.1.6 - Add filled(mem) constructor.
  • v0.1.5 - Change read_delimited to return Option<&[u8]>, for clean EOF handling.
  • v0.1.4 - Add clear().
  • v0.1.3
    • Thanks to freax13 for these changes:
      • Support any buffer size. Now you can make FixedBuf<[u8; 42]>.
      • Support any AsRef<[u8]> + AsMut<[u8]> value for internal memory:
        • [u8; N]
        • Box<[u8; N]>
        • &mut [u8]
        • Vec<u8>
    • Renamed new_with_mem to new. Use FixedBuf::default() to construct any FixedBuf<T: Default>, which includes arrays of sizes up to 32.
  • v0.1.2 - Updated documentation.
  • v0.1.1 - First published version

TODO