Module source

Module source 

Source
Expand description

Execution plan for reading Arrow IPC files

§Naming Note

The naming in this module can be confusing:

  • ArrowFileOpener handles the Arrow IPC file format (with footer, supports parallel reading)
  • ArrowStreamFileOpener handles the Arrow IPC stream format (without footer, sequential only)
  • ArrowSource is the unified FileSource implementation that uses either opener depending on the format specified at construction

Despite the name “ArrowStreamFileOpener”, it still reads from files - the “Stream” refers to the Arrow IPC stream format, not streaming I/O. Both formats can be stored in files on disk or object storage.

Structs§

ArrowOpener
FileOpener wrapper for both Arrow IPC file and stream formats
ArrowSource
FileSource for both Arrow IPC file and stream formats