pub struct MemoryMappedCsvReader { /* private fields */ }Expand description
Memory-mapped CSV reader for efficient processing of large files.
Implementations§
Source§impl MemoryMappedCsvReader
impl MemoryMappedCsvReader
pub fn new(path: &Path) -> Result<Self>
Sourcepub fn read_csv_chunk(
&self,
offset: u64,
chunk_size: usize,
has_headers: bool,
csv_config: Option<&CsvParserConfig>,
) -> Result<(Option<StringRecord>, Vec<StringRecord>, usize)>
pub fn read_csv_chunk( &self, offset: u64, chunk_size: usize, has_headers: bool, csv_config: Option<&CsvParserConfig>, ) -> Result<(Option<StringRecord>, Vec<StringRecord>, usize)>
Parse CSV records from memory-mapped data in chunks.
Returns (headers, records, actual_bytes_consumed).
Sourcepub fn read_csv_chunk_until_ragged(
&self,
offset: u64,
chunk_size: usize,
has_headers: bool,
csv_config: Option<&CsvParserConfig>,
) -> Result<(Option<StringRecord>, Vec<StringRecord>, usize, Option<Error>)>
pub fn read_csv_chunk_until_ragged( &self, offset: u64, chunk_size: usize, has_headers: bool, csv_config: Option<&CsvParserConfig>, ) -> Result<(Option<StringRecord>, Vec<StringRecord>, usize, Option<Error>)>
Self::read_csv_chunk, except that a record strict parsing refuses
for its field count ends the chunk instead of failing it: the records
before it are returned with the refusal. A caller under a row cap
raises it only if the cap is not reached first, so a record the
profile never reaches is never judged (#757).
Sourcepub fn ends_inside_quotes(
&self,
offset: u64,
csv_config: Option<&CsvParserConfig>,
) -> bool
pub fn ends_inside_quotes( &self, offset: u64, csv_config: Option<&CsvParserConfig>, ) -> bool
Whether the file ends inside a quoted field, scanning from offset on.
offset must be a record boundary, such as the offset of a chunk
Self::read_csv_chunk returned. Passing the last chunk’s offset keeps
the scan to that chunk. The CSV parser accepts a quote that is never
closed as a field running to the end of the file, so this is the only
sign of it.
Sourcepub fn estimate_csv_record_count(
&self,
csv_config: Option<&CsvParserConfig>,
) -> Result<usize>
pub fn estimate_csv_record_count( &self, csv_config: Option<&CsvParserConfig>, ) -> Result<usize>
Count logical CSV records, including the header, with a full scan. This shares the record parser with partial counting and avoids prefix bias.