Crate buf_ref_reader

Faster, growable buffering reader for when there’s little to no need to modify data, nor to keep it alive past next read.

std::io::BufReader works by copying data from its internal buffer into user-provided Vec/String, or, in case of .lines(), by emitting new heap-allocated String for each iteration. While convenient and versatile, this is not the fastest approach.

Instead, BufRefReader references its internal buffer with each read, returning &[u8]. Lack of extra allocations yields better read performance in situations where most (if not all) of read data:

• requires no modifications,
• is never used outside of a loop body and does not need to be duplicated into the heap for future use.

While being more performant, this approach also severely limits applicability of this reader:

• it does not (and cannot) implement BufRead and cannot be used as a direct replacement for BufReader;
• returned values are only valid between calls to reading functions (i.e. they cannot outlive even a single loop cycle), and Rust’s borrow checker will prevent you from using stale references;
• consequently, BufRefReader cannot be turned into an Iterator (here’s an easy way to think about it: what would Iterator::collect() return?);
• returned references are immutable;
• obviously, there’s also nothing that can return Strings or &strs for you.

Choice a of buffer

Use MmapBuffer unless:

• slice-deque is not available for your platform (e.g. no support for mmap),
• you need very small buffers (smaller than 1 memory page),
• you’re about to create a lot of buffers in a short period of time (new() is relatively expensive),
• you’re expecting buffer to grow a lot (consider, if possible, preallocating larger buffers through BufRefReaderBuilder.capacity),
• you have some very special concerns re: memory maps and malloc bypass (special allocators, possible kernel inefficiency due to large amount of mapped memory regions etc.).

Examples

Read data word by word:

use buf_ref_reader::*;

fn read<B: Buffer>() -> Result<(), Error>
where
	Error: From<B::Error>,
	// add this if you plan to `unwrap()` errors returned by `read()` et al.
	//B::Error: std::fmt::Debug,
{
	// &[u8] implements Read, hence we use it as our data source for this example
	let data = b"lorem ipsum dolor sit amet";
	let mut r = BufRefReaderBuilder::new(&data[..])
		.capacity(4)
		.build::<B>()?;

	assert_eq!(r.read_until(b' ')?, Some(&b"lorem "[..]));
	assert_eq!(r.read_until(b' ')?, Some(&b"ipsum "[..]));
	assert_eq!(r.read_until(b' ')?, Some(&b"dolor "[..]));
	assert_eq!(r.read_until(b' ')?, Some(&b"sit "[..]));
	assert_eq!(r.read_until(b' ')?, Some(&b"amet"[..]));
	assert_eq!(r.read_until(b' ')?, None); // EOF
	assert_eq!(r.read_until(b' ')?, None);

	Ok(())
}

fn main() {
	read::<VecBuffer>().unwrap();
	read::<MmapBuffer>().unwrap();
}

Structs

BufRefReader

Buffering reader.

BufRefReaderBuilder

Builder for BufRefReader.

MmapBuffer

Buffer that uses circular buffer implemented with mirrored memory maps

VecBuffer

Vec-backed buffer

Enums

Error

Error type that reading functions might emit

Traits

Buffer

This trait abstracts common operations with actual buffer from implementation details