Struct StreamingCsvProcessor 

pub struct StreamingCsvProcessor { /* private fields */ }

A streaming CSV processor that visits rows one at a time without materializing the full DataFrame.

Useful for large datasets where only aggregate statistics are needed. Memory usage is O(ncols) regardless of the number of rows.

Example

let csv = b"x,y\n1.0,2.0\n3.0,4.0\n5.0,6.0";
let mut proc = StreamingCsvProcessor::new(CsvConfig::default());
let (headers, sums, count) = proc.sum_columns(csv)?;

Implementations

impl StreamingCsvProcessor

pub fn new(config: CsvConfig) -> Self

pub fn sum_columns( &self, input: &[u8], ) -> Result<(Vec<String>, Vec<f64>, usize), DataError>

Stream through CSV, accumulating per-column sums using Kahan summation.

Returns (column_names, sums_per_col, row_count). Non-numeric fields contribute 0.0 to the sum.

pub fn minmax_columns( &self, input: &[u8], ) -> Result<(Vec<String>, Vec<f64>, Vec<f64>, usize), DataError>

Stream through CSV, collecting per-column min/max without materializing.

Returns (column_names, mins, maxs, row_count). Non-numeric fields contribute f64::NAN to min/max.