# fixed-buffer
[](https://crates.io/crates/fixed-buffer)
[](http://www.apache.org/licenses/LICENSE-2.0)
[](https://github.com/rust-secure-code/safety-dance/)
[](https://gitlab.com/leonhard-llc/fixed-buffer-rs/-/pipelines)
This is a Rust library with fixed-size buffers,
useful for network protocol parsers and file parsers.
# Features
- `forbid(unsafe_code)`
- Depends only on `std`
- Write bytes to the buffer and read them back
- Lives on the stack
- Does not allocate
- Use it to read a stream, search for a delimiter,
and save leftover bytes for the next read.
- No macros
- Good test coverage (100%)
- Async support by enabling cargo features `async-std-feature`, `futures-io`, `smol-feature`, or `tokio`.
# Limitations
- Not a circular buffer.
You can call `shift()` periodically
to move unread bytes to the front of the buffer.
# Examples
Read and handle requests from a remote client:
```rust
use fixed_buffer::{deframe_line, FixedBuf};
use std::io::Error;
use std::net::TcpStream;
fn handle_conn(mut tcp_stream: TcpStream) -> Result<(), Error> {
let mut buf: FixedBuf<4096> = FixedBuf::new();
loop {
// Read a line
// and leave leftover bytes in `buf`.
let line_bytes = match buf.read_frame(
&mut tcp_stream, deframe_line)? {
Some(line_bytes) => line_bytes,
None => return Ok(()),
};
let request = Request::parse(line_bytes)?;
handle_request(request)?;
}
}
```
For a complete example, see
[`tests/server.rs`](https://gitlab.com/leonhard-llc/fixed-buffer-rs/-/blob/main/fixed-buffer/tests/server.rs).
Read and process records:
```rust
use fixed_buffer::FixedBuf;
use std::io::{Error, ErrorKind, Read};
use std::net::TcpStream;
fn try_process_record(b: &[u8]) -> Result<usize, Error> {
if b.len() < 2 {
return Ok(0);
}
if b.starts_with("ab".as_bytes()) {
println!("found record");
Ok(2)
} else {
Err(Error::new(ErrorKind::InvalidData, "bad record"))
}
}
fn read_and_process<R: Read>(mut input: R) -> Result<(), Error> {
let mut buf: FixedBuf<1024> = FixedBuf::new();
loop {
// Read a chunk into the buffer.
if buf.copy_once_from(&mut input)? == 0 {
return if buf.len() == 0 {
// EOF at record boundary
Ok(())
} else {
// EOF in the middle of a record
Err(Error::from(
ErrorKind::UnexpectedEof))
};
}
// Process records in the buffer.
loop {
let num_to_consume =
try_process_record(buf.readable())?;
if num_to_consume == 0 {
break;
}
buf.try_read_exact(num_to_consume).unwrap();
}
// Shift data in the buffer to free up
// space at the end for writing.
buf.shift();
}
}
```
The `From<&[u8; SIZE]>` implementation is useful in tests. Example:
```rust
use core::convert::From;
assert_eq!(3, FixedBuf::from(b"abc").len());
```
# Cargo Geiger Safety Report
```
Metric output format: x/y
x = unsafe code used by the build
y = total unsafe code found in the crate
Symbols:
🔒 = No `unsafe` usage found, declares #![forbid(unsafe_code)]
❓ = No `unsafe` usage found, missing #![forbid(unsafe_code)]
☢️ = `unsafe` usage found
Functions Expressions Impls Traits Methods Dependency
0/0 0/0 0/0 0/0 0/0 🔒 fixed-buffer 1.0.2
0/0 0/0 0/0 0/0 0/0
```
# Alternatives
- [`bytes`](https://crates.io/crates/bytes), lots of [`unsafe`](https://github.com/search?q=repo%3Atokio-rs%2Fbytes+unsafe+path%3Asrc%2F**&type=code)
- [`buf_redux`](https://crates.io/crates/buf_redux), circular buffer support, updated in 2019
- [`std::io::BufReader`](https://doc.rust-lang.org/std/io/struct.BufReader.html)
- [`std::io::BufWriter`](https://doc.rust-lang.org/std/io/struct.BufWriter.html)
- [`static-buffer`](https://crates.io/crates/static-buffer), updated in 2016
- [`block-buffer`](https://crates.io/crates/block-buffer), for processing fixed-length blocks of data, some [`unsafe`](https://github.com/search?q=repo%3ARustCrypto%2Futils+unsafe+path%3Ablock-buffer%2F**&type=code)
- [`arrayvec`](https://crates.io/crates/arrayvec), vector with fixed capacity, some [`unsafe`](https://github.com/search?q=repo%3Abluss%2Farrayvec+unsafe+path%3Asrc%2F**&type=code)
# Changelog
- v1.0.2 2025-06-29 - Add `read_frame_tokio`.
- v1.0.1 2025-05-30 - Clarify docs.
- v1.0.0 2024-10-20 - `From<&[u8; SIZE]>`, `FixedBuffer::from(b"x")`
- v0.5.0 2022-03-21 - Move `ReadWriteChain` and `ReadWriteTake` to new
[`read-write-ext`](https://crates.io/crates/read-write-ext) crate.
- v0.4.0 2022-03-21
- `From<[u8; SIZE]>`, `FixedBuffer::from([0])`
- `write_bytes` to take `AsRef<[u8]>`
- Rename `try_read_exact` to `read_and_copy_exact`.
- Rename `try_read_bytes` to `try_read_exact`.
- Remove `empty`, `filled`, `read_byte`, `read_bytes`, `try_parse`, and `write_str`.
- `deframe` to allow consuming bytes without returning a frame
- `write_bytes` to write as many bytes as it can,
and return new `NoWritableSpace` error only when it cannot write any bytes.
Remove `NotEnoughSpaceError`. The method now behaves like `std::io::Write::write`.
<details>
<summary>Older changelog entries</summary>
- v0.3.1 - Implement `From<NotEnoughSpaceError>` and `From<MalformedInputError>` for `String`.
- v0.3.0 - Breaking API changes:
- Change type parameter to const buffer size. Example: `FixedBuf<1024>`.
- Remove `new` arg.
- Remove `capacity`.
- Remove `Copy` impl.
- Change `writable` return type to `&mut [u8]`.
- v0.2.3
- Add
[`read_byte`](https://docs.rs/fixed-buffer/latest/fixed_buffer/struct.FixedBuf.html#method.read_byte),
[`try_read_byte`](https://docs.rs/fixed-buffer/latest/fixed_buffer/struct.FixedBuf.html#method.try_read_byte),
[`try_read_bytes`](https://docs.rs/fixed-buffer/latest/fixed_buffer/struct.FixedBuf.html#method.try_read_bytes),
[`try_read_exact`](https://docs.rs/fixed-buffer/latest/fixed_buffer/struct.FixedBuf.html#method.try_read_exact),
[`try_parse`](https://docs.rs/fixed-buffer/latest/fixed_buffer/struct.FixedBuf.html#method.try_parse).
- Implement [`UnwindSafe`](https://doc.rust-lang.org/std/panic/trait.UnwindSafe.html)
- v0.2.2 - Add badges to readme
- v0.2.1 - Add
[`deframe`](https://docs.rs/fixed-buffer/latest/fixed_buffer/struct.FixedBuf.html#method.deframe)
and
[`mem`](https://docs.rs/fixed-buffer/latest/fixed_buffer/struct.FixedBuf.html#method.mem),
needed by `AsyncFixedBuf::read_frame`.
- v0.2.0
- Move tokio support to [`fixed_buffer_tokio`](https://crates.io/crates/fixed-buffer-tokio).
- Add
[`copy_once_from`](https://docs.rs/fixed-buffer/latest/fixed_buffer/struct.FixedBuf.html#method.copy_once_from),
[`read_block`](https://docs.rs/fixed-buffer/latest/fixed_buffer/struct.FixedBuf.html#method.read_block),
[`ReadWriteChain`](https://docs.rs/fixed-buffer/latest/fixed_buffer/struct.ReadWriteChain.html),
and
[`ReadWriteTake`](https://docs.rs/fixed-buffer/latest/fixed_buffer/struct.ReadWriteTake.html).
- v0.1.7 - Add [`FixedBuf::escape_ascii`](https://docs.rs/fixed-buffer/latest/fixed_buffer/struct.FixedBuf.html#method.escape_ascii).
- v0.1.6 - Add [`filled`](https://docs.rs/fixed-buffer/latest/fixed_buffer/struct.FixedBuf.html#method.filled)
constructor.
- v0.1.5 - Change [`read_delimited`](https://docs.rs/fixed-buffer/latest/fixed_buffer/struct.FixedBuf.html#method.read_delimited)
to return `Option<&[u8]>`, for clean EOF handling.
- v0.1.4 - Add [`clear()`](https://docs.rs/fixed-buffer/latest/fixed_buffer/struct.FixedBuf.html#method.clear).
- v0.1.3
- Thanks to [freax13](https://gitlab.com/Freax13) for these changes:
- Support any buffer size. Now you can make `FixedBuf<[u8; 42]>`.
- Support any `AsRef<[u8]> + AsMut<[u8]>` value for internal memory:
- `[u8; N]`
- `Box<[u8; N]>`
- `&mut [u8]`
- `Vec<u8>`
- Renamed `new_with_mem` to `new`.
Use `FixedBuf::default()` to construct any `FixedBuf<T: Default>`, which includes
[arrays of sizes up to 32](https://doc.rust-lang.org/std/primitive.array.html).
- v0.1.2 - Updated documentation.
- v0.1.1 - First published version
</details>
# TO DO
- Change links in docs to standard style. Don't link to `docs.rs`.
- Idea: `buf.slice(buf.read_frame(&mut reader, deframe_crlf))`
- Add an `frame_copy_iter` function.
Because of borrowing rules, this function must return non-borrowed (allocated and copied) data.
- Set up CI on:
- DONE - Linux x86 64-bit
- [macOS](https://gitlab.com/gitlab-org/gitlab/-/issues/269756)
- [Windows](https://about.gitlab.com/blog/2020/01/21/windows-shared-runner-beta/)
- <https://crate-ci.github.io/pr/testing.html#travisci>
- Linux ARM 64-bit (Raspberry Pi 3 and newer)
- Linux ARM 32-bit (Raspberry Pi 2)
- RISCV & ESP32 firmware?
- DONE - Try to make this crate comply with the [Rust API Guidelines](https://rust-lang.github.io/api-guidelines/).
- DONE - Find out how to include Readme.md info in the crate's docs.
- DONE - Make the repo public
- DONE - Set up continuous integration tests and banner.
- <https://github.com/actions-rs/example>
- <https://alican.codes/rust-github-actions/>
- DONE - Add some documentation tests
- <https://doc.rust-lang.org/rustdoc/documentation-tests.html>
- <https://doc.rust-lang.org/stable/rust-by-example/testing/doc_testing.html>
- DONE - Set up public repository on Gitlab.com
- <https://gitlab.com/mattdark/firebase-example/blob/master/.gitlab-ci.yml>
- <https://medium.com/astraol/optimizing-ci-cd-pipeline-for-rust-projects-gitlab-docker-98df64ae3bc4>
- <https://hub.docker.com/_/rust>
- DONE - Publish to crates.io
- DONE - Read through <https://crate-ci.github.io/index.html>
- DONE - Get a code review from an experienced rustacean
- DONE - Add and update a changelog
- Update it manually
- <https://crate-ci.github.io/release/changelog.html>
# Release Process
1. Edit `Cargo.toml` and bump version number.
2. Run `../release.sh`
License: Apache-2.0