Struct deltalake::datafusion::datasource::physical_plan::JsonOpener
source · pub struct JsonOpener { /* private fields */ }
Expand description
A FileOpener
that opens a JSON file and yields a FileOpenFuture
Implementations§
source§impl JsonOpener
impl JsonOpener
sourcepub fn new(
batch_size: usize,
projected_schema: Arc<Schema>,
file_compression_type: FileCompressionType,
object_store: Arc<dyn ObjectStore>
) -> JsonOpener
pub fn new( batch_size: usize, projected_schema: Arc<Schema>, file_compression_type: FileCompressionType, object_store: Arc<dyn ObjectStore> ) -> JsonOpener
Returns a JsonOpener
Trait Implementations§
source§impl FileOpener for JsonOpener
impl FileOpener for JsonOpener
source§fn open(
&self,
file_meta: FileMeta
) -> Result<Pin<Box<dyn Future<Output = Result<Pin<Box<dyn Stream<Item = Result<RecordBatch, ArrowError>> + Send>>, DataFusionError>> + Send>>, DataFusionError>
fn open( &self, file_meta: FileMeta ) -> Result<Pin<Box<dyn Future<Output = Result<Pin<Box<dyn Stream<Item = Result<RecordBatch, ArrowError>> + Send>>, DataFusionError>> + Send>>, DataFusionError>
Open a partitioned NDJSON file.
If file_meta.range
is None
, the entire file is opened.
Else file_meta.range
is Some(FileRange{start, end})
, which corresponds to the byte range [start, end) within the file.
Note: start
or end
might be in the middle of some lines. In such cases, the following rules
are applied to determine which lines to read:
- The first line of the partition is the line in which the index of the first character >=
start
. - The last line of the partition is the line in which the byte at position
end - 1
resides.
See CsvOpener
for an example.
Auto Trait Implementations§
impl Freeze for JsonOpener
impl !RefUnwindSafe for JsonOpener
impl Send for JsonOpener
impl Sync for JsonOpener
impl Unpin for JsonOpener
impl !UnwindSafe for JsonOpener
Blanket Implementations§
source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
source§impl<T> Instrument for T
impl<T> Instrument for T
source§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
source§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
source§impl<T> IntoEither for T
impl<T> IntoEither for T
source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
Converts
self
into a Left
variant of Either<Self, Self>
if into_left
is true
.
Converts self
into a Right
variant of Either<Self, Self>
otherwise. Read moresource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
Converts
self
into a Left
variant of Either<Self, Self>
if into_left(&self)
returns true
.
Converts self
into a Right
variant of Either<Self, Self>
otherwise. Read moreCreates a shared type from an unshared type.