Skip to main content

Module streaming

Module streaming 

Source
Expand description

Streaming record-by-record Arrow output for large files. Streaming record reader that produces Arrow RecordBatch objects

Reads fixed-length records from a byte reader and feeds them through the RecordBatchBuilder to produce batches.

Functionsยง

stream_to_batches
Stream binary records into Arrow RecordBatch objects.
stream_to_batches_with_schema
Stream binary records into Arrow RecordBatch objects, returning the Arrow schema as well.