[−][src]Crate fixed_buffer
This is a Rust library with fixed-size buffers, useful for network protocol parsers and file parsers.
Features
- No
unsafe
- Depends only on
std
- Write bytes to the buffer and read them back
- Lives on the stack
- Does not allocate memory
- Use it to read a stream, search for a delimiter, and save leftover bytes for the next read.
- Easy to learn & use. Easy to maintain code that uses it.
- Works with Rust
latest
,beta
, andnightly
- No macros
- Good test coverage (99%)
- fixed_buffer_tokio adds async functions
Limitations
- Not a circular buffer.
You can call
shift()
periodically to move unread bytes to the front of the buffer.
Documentation
https://docs.rs/fixed-buffer
Examples
Read and handle requests from a remote client:
use fixed_buffer::{ deframe_line, FixedBuf, ReadWriteChain}; use std::io::{Error, Read, Write}; use std::net::TcpStream; fn handle_request<RW: Read + Write>( reader_writer: &mut RW, request: Request, ) -> Result<(), Error> { // ... Ok(()) } fn handle_conn(mut tcp_stream: TcpStream ) -> Result<(), Error> { let mut buf: FixedBuf<[u8; 4096]> = FixedBuf::new([0; 4096]); loop { // Read a line // and leave leftover bytes in `buf`. let line_bytes = match buf.read_frame( &mut tcp_stream, deframe_line)? { Some(line_bytes) => line_bytes, None => return Ok(()), }; let request = Request::parse(line_bytes)?; // Read any request payload // from `buf` + `TcpStream`. let mut reader_writer = ReadWriteChain::new( &mut buf, &mut tcp_stream); handle_request(&mut reader_writer, request)?; } }
For a complete example, see
tests/server.rs
.
Read and process records:
use fixed_buffer::FixedBuf; use std::io::{Error, ErrorKind, Read}; use std::net::TcpStream; fn try_process_record(b: &[u8]) -> Result<usize, Error> { if b.len() < 2 { return Ok(0); } if b.starts_with("ab".as_bytes()) { println!("found record"); Ok(2) } else { Err(Error::new(ErrorKind::InvalidData, "bad record")) } } fn read_and_process<R: Read>(mut input: R) -> Result<(), Error> { let mut buf: FixedBuf<[u8; 1024]> = FixedBuf::new([0; 1024]); loop { // Read a chunk into the buffer. if buf.copy_once_from(&mut input)? == 0 { return if buf.len() == 0 { // EOF at record boundary Ok(()) } else { // EOF in the middle of a record Err(Error::from( ErrorKind::UnexpectedEof)) }; } // Process records in the buffer. loop { let bytes_read = try_process_record(buf.readable())?; if bytes_read == 0 { break; } buf.read_bytes(bytes_read); } // Shift data in the buffer to free up // space at the end for writing. buf.shift(); } }
The filled
constructor is useful in tests.
Alternatives
- bytes
- buf_redux, circular buffer support
- std::io::BufReader
- std::io::BufWriter
- static-buffer, updated in 2016
- block-buffer, for processing fixed-length blocks of data
- arrayvec, vector with fixed capacity.
Release Process
- Edit
Cargo.toml
and bump version number. - Run
../release.sh
Changelog
- v0.2.2 - Add badges to readme
- v0.2.1 - Add
deframe
andmem
, needed byAsyncFixedBuf::read_frame
. - v0.2.0
- Move tokio support to fixed_buffer_tokio.
- Add
copy_once_from
,read_block
,ReadWriteChain
, andReadWriteTake
.
- v0.1.7 - Add
FixedBuf::escape_ascii
. - v0.1.6 - Add
filled
constructor. - v0.1.5 - Change
read_delimited
to returnOption<&[u8]>
, for clean EOF handling. - v0.1.4 - Add
clear()
. - v0.1.3
- Thanks to freax13 for these changes:
- Support any buffer size. Now you can make
FixedBuf<[u8; 42]>
. - Support any
AsRef<[u8]> + AsMut<[u8]>
value for internal memory:[u8; N]
Box<[u8; N]>
&mut [u8]
Vec<u8>
- Support any buffer size. Now you can make
- Renamed
new_with_mem
tonew
. UseFixedBuf::default()
to construct anyFixedBuf<T: Default>
, which includes arrays of sizes up to 32.
- Thanks to freax13 for these changes:
- v0.1.2 - Updated documentation.
- v0.1.1 - First published version
TO DO
- DONE - Try to make this crate comply with the Rust API Guidelines.
- DONE - Find out how to include Readme.md info in the crate's docs.
- DONE - Make the repo public
- DONE - Set up continuous integration tests and banner.
- https://github.com/actions-rs/example
- https://alican.codes/rust-github-actions/
- DONE - Add some documentation tests
- https://doc.rust-lang.org/rustdoc/documentation-tests.html
- https://doc.rust-lang.org/stable/rust-by-example/testing/doc_testing.html
- DONE - Set up public repository on Gitlab.com
- https://gitlab.com/mattdark/firebase-example/blob/master/.gitlab-ci.yml
- https://medium.com/astraol/optimizing-ci-cd-pipeline-for-rust-projects-gitlab-docker-98df64ae3bc4
- https://hub.docker.com/_/rust
- DONE - Publish to creates.io
- DONE - Read through https://crate-ci.github.io/index.html
- DONE - Get a code review from an experienced rustacean
- DONE - Add and update a changelog
- Update it manually
- https://crate-ci.github.io/release/changelog.html
- Implement async-std read & write traits
- Add an
frame_copy_iter
function. Because of borrowing rules, this function must return non-borrowed (allocated and copied) data. - Switch to const generics once they are stable:
- https://github.com/rust-lang/rust/issues/44580
- https://stackoverflow.com/a/56543462
- Set up CI on:
Structs
FixedBuf | FixedBuf is a fixed-length byte buffer. You can write bytes to it and then read them back. |
MalformedInputError | |
NotEnoughSpaceError | |
ReadWriteChain | A wrapper for a pair of structs.
The first implements |
ReadWriteTake | Wraps a struct that implements
|
Functions
deframe_crlf | Deframes lines that are terminated by |
deframe_line | Deframes lines that are terminated by either |
escape_ascii | Convert a byte slice into a string. Includes printable ASCII characters as-is. Converts non-printable or non-ASCII characters to strings like "\n" and "\x19". |