Struct CsvToArrowReader 

pub struct CsvToArrowReader<R> { /* private fields */ }

CSV-to-Arrow streaming reader.

This struct reads CSV data from an async reader and converts it to Arrow RecordBatches. It supports configurable batch sizes and handles NULL values and type parsing.

Implementations

impl<R: AsyncBufRead + Unpin> CsvToArrowReader<R>

pub fn new(reader: R, schema: Arc<Schema>, options: &ArrowExportOptions) -> Self

where R: AsyncRead,

Creates a new CsvToArrowReader.

Arguments

• reader - The async reader to read CSV data from
• schema - The Arrow schema for the output RecordBatches
• options - Export options including batch size and NULL value

pub fn from_buffered( reader: BufReader<R>, schema: Arc<Schema>, options: &ArrowExportOptions, ) -> Self

where R: AsyncRead,

Creates a new CsvToArrowReader from an already buffered reader.

Arguments

• reader - The buffered async reader to read CSV data from
• schema - The Arrow schema for the output RecordBatches
• options - Export options including batch size and NULL value

pub fn schema(&self) -> Arc<Schema>

Returns the Arrow schema.

pub async fn next_batch(&mut self) -> Result<Option<RecordBatch>, ExportError>

Reads the next batch of rows and returns a RecordBatch.

Returns

• Ok(Some(batch)) - A RecordBatch with up to batch_size rows
• Ok(None) - No more data to read
• Err(e) - An error occurred during parsing or conversion