[−][src]Crate buf_ref_reader
Faster, growable buffering reader for when there's little to no need to modify data, nor to keep it alive past next read.
std::io::BufReader
works by copying data from its internal buffer into user-provided Vec
/String
,
or, in case of .lines()
, by emitting new heap-allocated String
for each iteration.
While convenient and versatile, this is not the fastest approach.
Instead, BufRefReader
references its internal buffer with each read, returning &[u8]
.
Lack of extra allocations yields better read performance in situations where most (if not all) of read data:
- requires no modifications,
- is never used outside of a loop body and does not need to be duplicated into the heap for future use.
While being more performant, this approach also severely limits applicability of this reader:
- it does not (and cannot) implement
BufRead
and cannot be used as a direct replacement forBufReader
; - returned values are only valid between calls to reading functions (i.e. they cannot outlive even a single loop cycle), and Rust's borrow checker will prevent you from using stale references;
- consequently,
BufRefReader
cannot be turned into anIterator
(here's an easy way to think about it: what wouldIterator::collect()
return?); - returned references are immutable;
- obviously, there's also nothing that can return
String
s or&str
s for you.
Choice a of buffer
Use MmapBuffer
unless:
- slice-deque is not available for your platform (e.g. no support for
mmap
), - you need very small buffers (smaller than 1 memory page),
- you're about to create a lot of buffers in a short period of time (
new()
is relatively expensive), - you're expecting buffer to grow a lot (consider, if possible, preallocating larger buffers through
BufRefReaderBuilder.capacity
), - you have some very special concerns re: memory maps and malloc bypass (special allocators, possible kernel inefficiency due to large amount of mapped memory regions etc.).
Examples
Read data word by word:
use buf_ref_reader::*; fn read<B: Buffer>() -> Result<(), Error> where Error: From<B::Error>, // add this if you plan to `unwrap()` errors returned by `read()` et al. //B::Error: std::fmt::Debug, { // &[u8] implements Read, hence we use it as our data source for this example let data = b"lorem ipsum dolor sit amet"; let mut r = BufRefReaderBuilder::new(&data[..]) .capacity(4) .build::<B>()?; assert_eq!(r.read_until(b' ')?, Some(&b"lorem "[..])); assert_eq!(r.read_until(b' ')?, Some(&b"ipsum "[..])); assert_eq!(r.read_until(b' ')?, Some(&b"dolor "[..])); assert_eq!(r.read_until(b' ')?, Some(&b"sit "[..])); assert_eq!(r.read_until(b' ')?, Some(&b"amet"[..])); assert_eq!(r.read_until(b' ')?, None); // EOF assert_eq!(r.read_until(b' ')?, None); Ok(()) } fn main() { read::<VecBuffer>().unwrap(); read::<MmapBuffer>().unwrap(); }
Structs
BufRefReader | Buffering reader. |
BufRefReaderBuilder | Builder for |
MmapBuffer | Buffer that uses circular buffer implemented with mirrored memory maps |
VecBuffer |
|
Enums
Error | Error type that reading functions might emit |
Traits
Buffer | This trait abstracts common operations with actual buffer from implementation details |