Crate fixed_buffer

source
Expand description

§fixed-buffer

crates.io version license: Apache 2.0 unsafe forbidden pipeline status

This is a Rust library with fixed-size buffers, useful for network protocol parsers and file parsers.

§Features

  • forbid(unsafe_code)
  • Depends only on std
  • Write bytes to the buffer and read them back
  • Lives on the stack
  • Does not allocate
  • Use it to read a stream, search for a delimiter, and save leftover bytes for the next read.
  • No macros
  • Good test coverage (100%)
  • Async support by enabling cargo features async-std-feature, futures-io, smol-feature, or tokio.

§Limitations

  • Not a circular buffer. You can call shift() periodically to move unread bytes to the front of the buffer.

§Examples

Read and handle requests from a remote client:

use fixed_buffer::{deframe_line, FixedBuf};
use std::io::Error;
use std::net::TcpStream;

fn handle_conn(mut tcp_stream: TcpStream) -> Result<(), Error> {
    let mut buf: FixedBuf<4096> = FixedBuf::new();
    loop {
        // Read a line
        // and leave leftover bytes in `buf`.
        let line_bytes = match buf.read_frame(
            &mut tcp_stream, deframe_line)? {
                Some(line_bytes) => line_bytes,
                None => return Ok(()),
            };
        let request = Request::parse(line_bytes)?;
        handle_request(request)?;
    }
}

For a complete example, see tests/server.rs.

Read and process records:

use fixed_buffer::FixedBuf;
use std::io::{Error, ErrorKind, Read};
use std::net::TcpStream;

fn try_process_record(b: &[u8]) -> Result<usize, Error> {
    if b.len() < 2 {
        return Ok(0);
    }
    if b.starts_with("ab".as_bytes()) {
        println!("found record");
        Ok(2)
    } else {
        Err(Error::new(ErrorKind::InvalidData, "bad record"))
    }
}

fn read_and_process<R: Read>(mut input: R) -> Result<(), Error> {
    let mut buf: FixedBuf<1024> = FixedBuf::new();
    loop {
        // Read a chunk into the buffer.
        if buf.copy_once_from(&mut input)? == 0 {
            return if buf.len() == 0 {
                // EOF at record boundary
                Ok(())
            } else {
                // EOF in the middle of a record
                Err(Error::from(
                    ErrorKind::UnexpectedEof))
            };
        }
        // Process records in the buffer.
        loop {
            let num_to_consume =
                try_process_record(buf.readable())?;
            if num_to_consume == 0 {
                break;
            }
            buf.try_read_exact(num_to_consume).unwrap();
        }
        // Shift data in the buffer to free up
        // space at the end for writing.
        buf.shift();
    }
}

The From<&[u8; SIZE]> implementation is useful in tests. Example:

use core::convert::From;
assert_eq!(3, FixedBuf::from(b"abc").len());

§Cargo Geiger Safety Report

§Alternatives

§Changelog

  • v1.0.0 2024-10-20 - From<&[u8; SIZE]>, FixedBuffer::from(b"x")
  • v0.5.0 2022-03-21 - Move ReadWriteChain and ReadWriteTake to new read-write-ext crate.
  • v0.4.0 2022-03-21
    • From<[u8; SIZE]>, FixedBuffer::from([0])
    • write_bytes to take AsRef<[u8]>
    • Rename try_read_exact to read_and_copy_exact.
    • Rename try_read_bytes to try_read_exact.
    • Remove empty, filled, read_byte, read_bytes, try_parse, and write_str.
    • deframe to allow consuming bytes without returning a frame
    • write_bytes to write as many bytes as it can, and return new NoWritableSpace error only when it cannot write any bytes. Remove NotEnoughSpaceError. The method now behaves like std::io::Write::write.
Older changelog entries
  • v0.3.1 - Implement From<NotEnoughSpaceError> and From<MalformedInputError> for String.
  • v0.3.0 - Breaking API changes:
    • Change type parameter to const buffer size. Example: FixedBuf<1024>.
    • Remove new arg.
    • Remove capacity.
    • Remove Copy impl.
    • Change writable return type to &mut [u8].
  • v0.2.3
  • v0.2.2 - Add badges to readme
  • v0.2.1 - Add deframe and mem, needed by AsyncFixedBuf::read_frame.
  • v0.2.0
  • v0.1.7 - Add FixedBuf::escape_ascii.
  • v0.1.6 - Add filled constructor.
  • v0.1.5 - Change read_delimited to return Option<&[u8]>, for clean EOF handling.
  • v0.1.4 - Add clear().
  • v0.1.3
    • Thanks to freax13 for these changes:
      • Support any buffer size. Now you can make FixedBuf<[u8; 42]>.
      • Support any AsRef<[u8]> + AsMut<[u8]> value for internal memory:
        • [u8; N]
        • Box<[u8; N]>
        • &mut [u8]
        • Vec<u8>
    • Renamed new_with_mem to new. Use FixedBuf::default() to construct any FixedBuf<T: Default>, which includes arrays of sizes up to 32.
  • v0.1.2 - Updated documentation.
  • v0.1.1 - First published version

§TO DO

§Release Process

  1. Edit Cargo.toml and bump version number.
  2. Run ../release.sh

Structs§

Functions§

  • deframe_crlf
    Deframes lines that are terminated by b"\r\n". Ignores solitary b'\n'.
  • deframe_line
    Deframes lines that are terminated by either b'\n' or b"\r\n".
  • escape_ascii
    Convert a byte slice into a string. Includes printable ASCII characters as-is. Converts non-printable or non-ASCII characters to strings like “\n” and “\x19”.