[][src]Crate buf_ref_reader

Faster, growable buffering reader for when there's little to no need to modify data, nor to keep it alive past next read.

std::io::BufReader works by copying data from its internal buffer into user-provided Vec/String, or, in case of .lines(), by emitting new heap-allocated String for each iteration. While convenient and versatile, this is not the fastest approach.

Instead, BufRefReader references its internal buffer with each read, returning &[u8]. Lack of extra allocations yields better read performance in situations where most (if not all) of read data:

  • requires no modifications,
  • is never used outside of a loop body and does not need to be duplicated into the heap for future use.

While being more performant, this approach also severely limits applicability of this reader:

  • it does not (and cannot) implement BufRead and cannot be used as a direct replacement for BufReader;
  • returned values are only valid between calls to reading functions (i.e. they cannot outlive even a single loop cycle), and Rust's borrow checker will prevent you from using stale references;
  • consequently, BufRefReader cannot be turned into an Iterator (here's an easy way to think about it: what would Iterator::collect() return?);
  • returned references are immutable;
  • obviously, there's also nothing that can return Strings or &strs for you.

Choice a of buffer

Use MmapBuffer unless:

  • slice-deque is not available for your platform (e.g. no support for mmap),
  • you need very small buffers (smaller than 1 memory page),
  • you're about to create a lot of buffers in a short period of time (new() is relatively expensive),
  • you're expecting buffer to grow a lot (consider, if possible, preallocating larger buffers through BufRefReaderBuilder.capacity),
  • you have some very special concerns re: memory maps and malloc bypass (special allocators, possible kernel inefficiency due to large amount of mapped memory regions etc.).


Read data word by word:

use buf_ref_reader::*;

fn read<B: Buffer>() -> Result<(), Error>
    Error: From<B::Error>,
    // add this if you plan to `unwrap()` errors returned by `read()` et al.
    //B::Error: std::fmt::Debug,
    // &[u8] implements Read, hence we use it as our data source for this example
    let data = b"lorem ipsum dolor sit amet";
    let mut r = BufRefReaderBuilder::new(&data[..])

    assert_eq!(r.read_until(b' ')?, Some(&b"lorem "[..]));
    assert_eq!(r.read_until(b' ')?, Some(&b"ipsum "[..]));
    assert_eq!(r.read_until(b' ')?, Some(&b"dolor "[..]));
    assert_eq!(r.read_until(b' ')?, Some(&b"sit "[..]));
    assert_eq!(r.read_until(b' ')?, Some(&b"amet"[..]));
    assert_eq!(r.read_until(b' ')?, None); // EOF
    assert_eq!(r.read_until(b' ')?, None);


fn main() {



Buffering reader.


Builder for BufRefReader.


Buffer that uses circular buffer implemented with mirrored memory maps


Vec-backed buffer



Error type that reading functions might emit



This trait abstracts common operations with actual buffer from implementation details