lines - streaming through text line by line
lines
is a small library to efficiently parse text files line by
line while avoiding unnecessary memory allocations.
Typically, processing log files or other line oriented file formats, doesn't always require allocation of memory for each processed line. Instead, we can ...
- ... allocate and maintain one buffer ...
- ... locate a line in the processed stream and copy that to the mentioned buffer ...
- ... lend the buffer to client code for "some number crunching," and ...
- ... afterwards repeat locating the next line in the input at step 2.
By re-using a buffer and having the client decide whether or not to make a copy of the read line, we can gain significant performance wins in certain situations.
Usage
Since lines
uses Cargo, adding a dependency
section as follows should suffice to start using it:
[]
= "*"
The typical example of iterating a file line by line can be demonstrated with the following program:
extern crate lines;
use LineReader;
use File;
use from_utf8;
There are certain limitations to the data that the library can process. Namely, a newline is assumed to be defined by '\n'. For more information can be found in the generated documentation of the library.