Struct parquet::arrow::async_reader::ParquetObjectReader
source · pub struct ParquetObjectReader { /* private fields */ }
Expand description
Reads Parquet files in object storage using ObjectStore
.
// Populate configuration from environment
let storage_container = Arc::new(MicrosoftAzureBuilder::from_env().build().unwrap());
let location = Path::from("path/to/blob.parquet");
let meta = storage_container.head(&location).await.unwrap();
println!("Found Blob with {}B at {}", meta.size, meta.location);
// Show Parquet metadata
let reader = ParquetObjectReader::new(storage_container, meta);
let builder = ParquetRecordBatchStreamBuilder::new(reader).await.unwrap();
print_parquet_metadata(&mut stdout(), builder.metadata());
Implementations§
source§impl ParquetObjectReader
impl ParquetObjectReader
sourcepub fn new(store: Arc<dyn ObjectStore>, meta: ObjectMeta) -> Self
pub fn new(store: Arc<dyn ObjectStore>, meta: ObjectMeta) -> Self
Creates a new ParquetObjectReader
for the provided ObjectStore
and ObjectMeta
ObjectMeta
can be obtained using ObjectStore::list
or ObjectStore::head
Provide a hint as to the size of the parquet file’s footer, see fetch_parquet_metadata
sourcepub fn with_preload_column_index(self, preload_column_index: bool) -> Self
pub fn with_preload_column_index(self, preload_column_index: bool) -> Self
Load the Column Index as part of Self::get_metadata
sourcepub fn with_preload_offset_index(self, preload_offset_index: bool) -> Self
pub fn with_preload_offset_index(self, preload_offset_index: bool) -> Self
Load the Offset Index as part of Self::get_metadata
Trait Implementations§
source§impl AsyncFileReader for ParquetObjectReader
impl AsyncFileReader for ParquetObjectReader
source§fn get_bytes(&mut self, range: Range<usize>) -> BoxFuture<'_, Result<Bytes>>
fn get_bytes(&mut self, range: Range<usize>) -> BoxFuture<'_, Result<Bytes>>
Retrieve the bytes in
range
source§fn get_byte_ranges(
&mut self,
ranges: Vec<Range<usize>>
) -> BoxFuture<'_, Result<Vec<Bytes>>>where
Self: Send,
fn get_byte_ranges(
&mut self,
ranges: Vec<Range<usize>>
) -> BoxFuture<'_, Result<Vec<Bytes>>>where
Self: Send,
Retrieve multiple byte ranges. The default implementation will call
get_bytes
sequentiallysource§fn get_metadata(&mut self) -> BoxFuture<'_, Result<Arc<ParquetMetaData>>>
fn get_metadata(&mut self) -> BoxFuture<'_, Result<Arc<ParquetMetaData>>>
Provides asynchronous access to the
ParquetMetaData
of a parquet file,
allowing fine-grained control over how metadata is sourced, in particular allowing
for caching, pre-fetching, catalog metadata, etc…source§impl Clone for ParquetObjectReader
impl Clone for ParquetObjectReader
source§fn clone(&self) -> ParquetObjectReader
fn clone(&self) -> ParquetObjectReader
Returns a copy of the value. Read more
1.0.0 · source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
Performs copy-assignment from
source
. Read moreAuto Trait Implementations§
impl !RefUnwindSafe for ParquetObjectReader
impl Send for ParquetObjectReader
impl Sync for ParquetObjectReader
impl Unpin for ParquetObjectReader
impl !UnwindSafe for ParquetObjectReader
Blanket Implementations§
source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more