Struct JsonOpener 

pub struct JsonOpener { /* private fields */ }

A FileOpener that opens a JSON file and yields a FileOpenFuture

Implementations

impl JsonOpener

pub fn new( batch_size: usize, projected_schema: SchemaRef, file_compression_type: FileCompressionType, object_store: Arc<dyn ObjectStore>, ) -> Self

Returns a JsonOpener

Trait Implementations

impl FileOpener for JsonOpener

fn open(&self, partitioned_file: PartitionedFile) -> Result<FileOpenFuture>

Open a partitioned NDJSON file.

If file_meta.range is None, the entire file is opened. Else file_meta.range is Some(FileRange{start, end}), which corresponds to the byte range [start, end) within the file.

Note: start or end might be in the middle of some lines. In such cases, the following rules are applied to determine which lines to read:

1. The first line of the partition is the line in which the index of the first character >= start.
2. The last line of the partition is the line in which the byte at position end - 1 resides.