Struct lance_core::io::reader::FileReader

pub struct FileReader { /* private fields */ }

Lance File Reader.

It reads arrow data from one data file.

Implementations

impl FileReader

pub async fn try_new_with_fragment( object_store: &ObjectStore, path: &Path, fragment_id: u64, manifest: Option<&Manifest>, session: Option<&FileMetadataCache> ) -> Result<Self>

Open file reader

Open the file at the given path using the provided object store.

The passed fragment ID determines the first 32-bits of the row IDs.

If a manifest is passed in, it will be used to load the schema and dictionary. This is typically done if the file is part of a dataset fragment. If no manifest is passed in, then it is read from the file itself.

The session passed in is used to cache metadata about the file. If no session is passed in, there will be no caching.

pub async fn try_new(object_store: &ObjectStore, path: &Path) -> Result<Self>

Open one Lance data file for read.

pub fn with_row_id(&mut self, v: bool) -> &mut Self

Instruct the FileReader to return meta row id column.

pub fn with_make_deletions_null(&mut self, val: bool) -> &mut Self

Instruct the FileReader that instead of removing deleted rows, it may simply mark the _rowid value as null. Some rows may still be removed, for example if the entire batch is deleted. This is a performance optimization where the null bitmap of the _rowid column serves as a selection vector.

pub fn schema(&self) -> &Schema

Schema of the returning RecordBatch.

pub fn num_batches(&self) -> usize

pub fn num_rows_in_batch(&self, batch_id: i32) -> usize

Get the number of rows in this batch

pub fn len(&self) -> usize

Count the number of rows in this file.

pub fn is_empty(&self) -> bool

pub async fn read_batch( &self, batch_id: i32, params: impl Into<ReadBatchParams>, projection: &Schema ) -> Result<RecordBatch>

Read a batch of data from the file.

The schema of the returned RecordBatch is set by FileReader::schema().

pub async fn read_range( &self, range: Range<usize>, projection: &Schema ) -> Result<RecordBatch>

Read a range of records into one batch.

Note that it might call concat if the range is crossing multiple batches, which makes it less efficient than FileReader::read_batch().

pub async fn take( &self, indices: &[u32], projection: &Schema ) -> Result<RecordBatch>

Take by records by indices within the file.

The indices must be sorted.

pub async fn read_page_stats( &self, projection: &Schema ) -> Result<Option<RecordBatch>>

Trait Implementations

impl Clone for FileReader

fn clone(&self) -> FileReader

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for FileReader

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.