Crate fixed_buffer[−][src]

Expand description

crates.io version license: Apache 2.0 unsafe forbidden pipeline status

This is a Rust library with fixed-size buffers, useful for network protocol parsers and file parsers.

Features

  • No unsafe
  • Depends only on std
  • Write bytes to the buffer and read them back
  • Lives on the stack
  • Does not allocate memory
  • Use it to read a stream, search for a delimiter, and save leftover bytes for the next read.
  • Easy to learn & use. Easy to maintain code that uses it.
  • Works with Rust latest, beta, and nightly
  • No macros
  • Good test coverage (99%)
  • fixed_buffer_tokio adds async functions

Limitations

  • Not a circular buffer. You can call shift() periodically to move unread bytes to the front of the buffer.

Documentation

https://docs.rs/fixed-buffer

Examples

Read and handle requests from a remote client:

use fixed_buffer::{
    deframe_line, FixedBuf, ReadWriteChain};
use std::io::{Error, Read, Write};
use std::net::TcpStream;

fn handle_request<RW: Read + Write>(
    reader_writer: &mut RW,
    request: Request,
) -> Result<(), Error> {
    // ...
    Ok(())
}

fn handle_conn(mut tcp_stream: TcpStream
) -> Result<(), Error> {
    let mut buf: FixedBuf<4096> = FixedBuf::new();
    loop {
        // Read a line
        // and leave leftover bytes in `buf`.
        let line_bytes = match buf.read_frame(
            &mut tcp_stream, deframe_line)? {
                Some(line_bytes) => line_bytes,
                None => return Ok(()),
            };
        let request = Request::parse(line_bytes)?;
        // Read any request payload
        // from `buf` + `TcpStream`.
        let mut reader_writer = ReadWriteChain::new(
            &mut buf, &mut tcp_stream);
        handle_request(&mut reader_writer, request)?;
    }
}

For a complete example, see tests/server.rs.

Read and process records:

use fixed_buffer::FixedBuf;
use std::io::{Error, ErrorKind, Read};
use std::net::TcpStream;

fn try_process_record(b: &[u8]) -> Result<usize, Error> {
    if b.len() < 2 {
        return Ok(0);
    }
    if b.starts_with("ab".as_bytes()) {
        println!("found record");
        Ok(2)
    } else {
        Err(Error::new(ErrorKind::InvalidData, "bad record"))
    }
}

fn read_and_process<R: Read>(mut input: R)
    -> Result<(), Error> {
    let mut buf: FixedBuf<1024> = FixedBuf::new();
    loop {
        // Read a chunk into the buffer.
        if buf.copy_once_from(&mut input)? == 0 {
            return if buf.len() == 0 {
                // EOF at record boundary
                Ok(())
            } else {
                // EOF in the middle of a record
                Err(Error::from(
                    ErrorKind::UnexpectedEof))
            };
        }
        // Process records in the buffer.
        loop {
            let bytes_read =
                try_process_record(buf.readable())?;
            if bytes_read == 0 {
                break;
            }
            buf.read_bytes(bytes_read);
        }
        // Shift data in the buffer to free up
        // space at the end for writing.
        buf.shift();
    }
}

The filled constructor is useful in tests.

Alternatives

Changelog

TO DO

  • Change deframer function type to allow consuming bytes without returning a block.
  • Implement async-std read & write traits
  • Add an frame_copy_iter function. Because of borrowing rules, this function must return non-borrowed (allocated and copied) data.
  • Set up CI on:
    • DONE - Linux x86 64-bit
    • macOS
    • Windows
    • https://crate-ci.github.io/pr/testing.html#travisci
    • Linux ARM 64-bit (Raspberry Pi 3 and newer)
    • Linux ARM 32-bit (Raspberry Pi 2)
    • RISCV & ESP32 firmware?
  • DONE - Switch to const generics once they are stable:
    • https://github.com/rust-lang/rust/issues/44580
    • https://stackoverflow.com/a/56543462
  • DONE - Try to make this crate comply with the Rust API Guidelines.
  • DONE - Find out how to include Readme.md info in the crate’s docs.
  • DONE - Make the repo public
  • DONE - Set up continuous integration tests and banner.
    • https://github.com/actions-rs/example
    • https://alican.codes/rust-github-actions/
  • DONE - Add some documentation tests
    • https://doc.rust-lang.org/rustdoc/documentation-tests.html
    • https://doc.rust-lang.org/stable/rust-by-example/testing/doc_testing.html
  • DONE - Set up public repository on Gitlab.com
    • https://gitlab.com/mattdark/firebase-example/blob/master/.gitlab-ci.yml
    • https://medium.com/astraol/optimizing-ci-cd-pipeline-for-rust-projects-gitlab-docker-98df64ae3bc4
    • https://hub.docker.com/_/rust
  • DONE - Publish to creates.io
  • DONE - Read through https://crate-ci.github.io/index.html
  • DONE - Get a code review from an experienced rustacean
  • DONE - Add and update a changelog
    • Update it manually
    • https://crate-ci.github.io/release/changelog.html

Release Process

  1. Edit Cargo.toml and bump version number.
  2. Run ../release.sh

Structs

FixedBuf

FixedBuf is a fixed-length byte buffer. You can write bytes to it and then read them back.

MalformedInputError
NotEnoughSpaceError
ReadWriteChain

A wrapper for a pair of structs. The first implements Read. The second implements Read+Write.

ReadWriteTake

Wraps a struct that implements Read+Write. Passes through reads and writes to the struct. Limits the number of bytes that can be read.

Functions

deframe_crlf

Deframes lines that are terminated by b"\r\n". Ignores solitary b'\n'.

deframe_line

Deframes lines that are terminated by either b'\n' or b"\r\n".

escape_ascii

Convert a byte slice into a string. Includes printable ASCII characters as-is. Converts non-printable or non-ASCII characters to strings like “\n” and “\x19”.