Struct ArrowEngineData 

pub struct ArrowEngineData { /* private fields */ }

ArrowEngineData holds an Arrow RecordBatch, implements EngineData so the kernel can extract from it.

WARNING: Row visitors require that all leaf columns of the record batch have correctly computed NULL masks. The arrow parquet reader is known to produce incomplete NULL masks, for example. When in doubt, call fix_nested_null_masks first.

Implementations

impl ArrowEngineData

pub fn new(data: RecordBatch) -> ArrowEngineData

Create a new ArrowEngineData from a RecordBatch

pub fn try_from_engine_data( engine_data: Box<dyn EngineData>, ) -> Result<Box<ArrowEngineData>, Error>

Utility constructor to get a Box<ArrowEngineData> out of a Box<dyn EngineData>

pub fn record_batch(&self) -> &RecordBatch

Get a reference to the RecordBatch this ArrowEngineData is wrapping

Trait Implementations

impl EngineData for ArrowEngineData

fn len(&self) -> usize

Return the number of items (rows) in blob

fn visit_rows( &self, leaf_columns: &[ColumnName], visitor: &mut dyn RowVisitor, ) -> Result<(), Error>

Visits a subset of leaf columns in each row of this data, passing a GetData item for each requested column to the visitor’s visit method (along with the number of rows of data to be visited).

fn append_columns( &self, schema: Arc<StructType>, columns: Vec<ArrayData>, ) -> Result<Box<dyn EngineData>, Error>

Append new columns provided by Kernel to the existing data.

fn apply_selection_vector( self: Box<ArrowEngineData>, selection_vector: Vec<bool>, ) -> Result<Box<dyn EngineData>, Error>

Apply a selection vector to the data and return a data where only the valid rows are included. This consumes the EngineData, allowing engines to implement this “in place” if desired

fn is_empty(&self) -> bool

Returns true if the data is empty (i.e., has no rows).

impl From<RecordBatch> for ArrowEngineData

fn from(value: RecordBatch) -> ArrowEngineData

Converts to this type from the input type.

impl From<StructArray> for ArrowEngineData

fn from(value: StructArray) -> ArrowEngineData

Converts to this type from the input type.