Struct parquet::arrow::async_reader::ParquetObjectReader
source · pub struct ParquetObjectReader { /* private fields */ }
Expand description
Reads Parquet files in object storage using ObjectStore
.
// Populate configuration from environment
let storage_container = Arc::new(MicrosoftAzureBuilder::from_env().build().unwrap());
let location = Path::from("path/to/blob.parquet");
let meta = storage_container.head(&location).await.unwrap();
println!("Found Blob with {}B at {}", meta.size, meta.location);
// Show Parquet metadata
let reader = ParquetObjectReader::new(storage_container, meta);
let builder = ParquetRecordBatchStreamBuilder::new(reader).await.unwrap();
print_parquet_metadata(&mut stdout(), builder.metadata());
Implementations§
source§impl ParquetObjectReader
impl ParquetObjectReader
sourcepub fn new(store: Arc<dyn ObjectStore>, meta: ObjectMeta) -> Self
pub fn new(store: Arc<dyn ObjectStore>, meta: ObjectMeta) -> Self
Creates a new ParquetObjectReader
for the provided ObjectStore
and ObjectMeta
ObjectMeta
can be obtained using ObjectStore::list
or ObjectStore::head
Provide a hint as to the size of the parquet file’s footer, see fetch_parquet_metadata
sourcepub fn with_preload_column_index(self, preload_column_index: bool) -> Self
pub fn with_preload_column_index(self, preload_column_index: bool) -> Self
Load the Column Index as part of Self::get_metadata
sourcepub fn with_preload_offset_index(self, preload_offset_index: bool) -> Self
pub fn with_preload_offset_index(self, preload_offset_index: bool) -> Self
Load the Offset Index as part of Self::get_metadata
Trait Implementations§
source§impl AsyncFileReader for ParquetObjectReader
impl AsyncFileReader for ParquetObjectReader
source§fn get_bytes(&mut self, range: Range<usize>) -> BoxFuture<'_, Result<Bytes>>
fn get_bytes(&mut self, range: Range<usize>) -> BoxFuture<'_, Result<Bytes>>
Retrieve the bytes in
range
source§fn get_byte_ranges(
&mut self,
ranges: Vec<Range<usize>>
) -> BoxFuture<'_, Result<Vec<Bytes>>>where
Self: Send,
fn get_byte_ranges(
&mut self,
ranges: Vec<Range<usize>>
) -> BoxFuture<'_, Result<Vec<Bytes>>>where
Self: Send,
Retrieve multiple byte ranges. The default implementation will call
get_bytes
sequentiallysource§fn get_metadata(&mut self) -> BoxFuture<'_, Result<Arc<ParquetMetaData>>>
fn get_metadata(&mut self) -> BoxFuture<'_, Result<Arc<ParquetMetaData>>>
Provides asynchronous access to the
ParquetMetaData
of a parquet file,
allowing fine-grained control over how metadata is sourced, in particular allowing
for caching, pre-fetching, catalog metadata, etc…source§impl Clone for ParquetObjectReader
impl Clone for ParquetObjectReader
source§fn clone(&self) -> ParquetObjectReader
fn clone(&self) -> ParquetObjectReader
Returns a copy of the value. Read more
1.0.0 · source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
Performs copy-assignment from
source
. Read moreAuto Trait Implementations§
impl Freeze for ParquetObjectReader
impl !RefUnwindSafe for ParquetObjectReader
impl Send for ParquetObjectReader
impl Sync for ParquetObjectReader
impl Unpin for ParquetObjectReader
impl !UnwindSafe for ParquetObjectReader
Blanket Implementations§
source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more