Skip to main content

Crate csveee

Crate csveee 

Source
Expand description

A very fast, parallel CSV parser.

An input is split into chunks that are parsed concurrently, each into a state of its own, and a merge function folds the per-chunk states into the result. Fields are handed to the callback as mutable slices into the parser’s own buffer, so no per-record allocation happens.

use csveee::Parser;

let mut parser = Parser::new();
let cities = parser.parse(
    "data.csv",
    Vec::new,
    |acc, [_name, _age, city]| {
        acc.push(city.to_string());
        Ok(())
    },
    |states| states.concat(),
)?;

Parser::parse_slice does the same for bytes already in memory, and Parser::parse_stream is the sequential fallback for sources without random access. ParserBuilder configures the dialect (delimiter, quoting, comments), the output mode (Variadic for slices instead of fixed-size arrays, Bytes to skip UTF-8 validation) and the execution and I/O backends.

§Side effects

A chunk’s starting state — inside a quoted field or outside it — is a guess, and a wrong one is retried, so the accumulator may run over a chunk more than once. A wrong guess cuts the same bytes into different fields and records; that pass is dropped at merge time, and a chunk that must be reparsed accumulates a second time. Only what merge folds matches the file; an effect the accumulator has outside the state it is handed — a print, a shared counter — is not undone. A one-chunk input never speculates, so small tests hide this. Parser::parse_stream is sequential and sees every record once.

Structs§

Arity
Marker type: fields are handed to the callback as a fixed-size array [&mut T; N] (default).
Parser
The CSV parser, parameterized by field count mode and encoding.
ParserBuilder
A CSV parser builder.
Position
The location of a parser error in the input.
RingBufSettings
Per-thread ring-buffer I/O settings.
Variadic
Marker type: fields are handed to the callback as a slice &mut [&mut T].

Enums§

Error
A CSV parser error.
IoBackend
The I/O backend to use for reading file data.
ParserBackend
The chunk parser implementation to use.
QuoteHandling
How quote characters are handled.
RecordTerminator
The record terminator.

Type Aliases§

Bytes
Fields are raw bytes, no UTF-8 validation. For non-UTF-8 encodings.
Result
A Result with this crate’s Error type.
Text
Fields are validated as UTF-8 and handed out as &mut str (default).