Expand description
A very fast, parallel CSV parser.
An input is split into chunks that are parsed concurrently, each into a state of its own, and a merge function folds the per-chunk states into the result. Fields are handed to the callback as mutable slices into the parser’s own buffer, so no per-record allocation happens.
use csveee::Parser;
let mut parser = Parser::new();
let cities = parser.parse(
"data.csv",
Vec::new,
|acc, [_name, _age, city]| {
acc.push(city.to_string());
Ok(())
},
|states| states.concat(),
)?;Parser::parse_slice does the same for bytes already in memory,
and Parser::parse_stream is the sequential fallback for sources
without random access. ParserBuilder configures the dialect
(delimiter, quoting, comments), the output mode (Variadic for
slices instead of fixed-size arrays, Bytes to skip UTF-8
validation) and the execution and I/O backends.
§Side effects
A chunk’s starting state — inside a quoted field or outside it — is
a guess, and a wrong one is retried, so the accumulator may run over
a chunk more than once. A wrong guess cuts the same bytes into
different fields and records; that pass is dropped at merge time,
and a chunk that must be reparsed accumulates a second time. Only
what merge folds matches the file; an effect the accumulator has
outside the state it is handed — a print, a shared counter — is not
undone. A one-chunk input never speculates, so small tests hide
this.
Parser::parse_stream is sequential and sees every record once.
Structs§
- Arity
- Marker type: fields are handed to the callback as a fixed-size array
[&mut T; N](default). - Parser
- The CSV parser, parameterized by field count mode and encoding.
- Parser
Builder - A CSV parser builder.
- Position
- The location of a parser error in the input.
- Ring
BufSettings - Per-thread ring-buffer I/O settings.
- Variadic
- Marker type: fields are handed to the callback as a slice
&mut [&mut T].
Enums§
- Error
- A CSV parser error.
- IoBackend
- The I/O backend to use for reading file data.
- Parser
Backend - The chunk parser implementation to use.
- Quote
Handling - How quote characters are handled.
- Record
Terminator - The record terminator.