Struct QueryHandle 

pub struct QueryHandle<E: StorageEngineLike> { /* private fields */ }

A handle to a dataframe query, ready to be executed.

Cheaply created via QueryEngine::query.

See QueryHandle::next_row or QueryHandle::into_iter.

Implementations

impl<E: StorageEngineLike> QueryHandle<E>

pub fn query(&self) -> &QueryExpression

The query used to instantiate this handle.

pub fn view_contents(&self) -> &ChunkColumnDescriptors

Describes the columns that make up this view.

See QueryExpression::view_contents.

pub fn selected_contents(&self) -> &[(usize, ColumnDescriptor)]

Describes the columns that make up this selection.

The extra usize is the index in Self::view_contents that this selection points to.

See QueryExpression::selection.

pub fn schema(&self) -> &ArrowSchemaRef

All results returned by this handle will strictly follow this Arrow schema.

Columns that do not yield any data will still be present in the results, filled with null values.

pub fn seek_to_row(&mut self, row_idx: usize)

Advance all internal cursors so that the next row yielded will correspond to row_idx.

Does nothing if row_idx is out of bounds.

Performance

This requires going through every chunk once, and for each chunk running a binary search if the chunk’s time range contains the index_value.

I.e.: it’s pretty cheap already.

pub fn num_rows(&self) -> u64

How many rows of data will be returned?

The number of rows depends and only depends on the view contents. The selected contents has no influence on this value.

pub fn row_index_at_or_before_time(&self, time: TimeInt) -> Option<u64>

Returns the row index of the last row whose index value is <= the given time, or None if no such row exists.

pub fn next_row(&mut self) -> Option<Vec<ArrayRef>>

Returns the next row’s worth of data.

The returned vector of Arrow arrays strictly follows the schema specified by Self::schema. Columns that do not yield any data will still be present in the results, filled with null values.

Each cell in the result corresponds to the latest locally known value at that particular point in the index, for each respective ColumnDescriptor. See QueryExpression::sparse_fill_strategy to go beyond local resolution.

Example:

while let Some(row) = query_handle.next_row() {
    // …
}

Pagination

Use Self::seek_to_row:

query_handle.seek_to_row(42);
for row in query_handle.into_iter().take(len) {
    // …
}

pub fn next_row_async( &mut self, ) -> impl Future<Output = Option<Vec<ArrayRef>>> + use<E>

where E: 'static + Send + Clone,

Asynchronously returns the next row’s worth of data.

The returned vector of Arrow arrays strictly follows the schema specified by Self::schema. Columns that do not yield any data will still be present in the results, filled with null values.

Each cell in the result corresponds to the latest locally known value at that particular point in the index, for each respective ColumnDescriptor. See QueryExpression::sparse_fill_strategy to go beyond local resolution.

Example:

while let Some(row) = query_handle.next_row_async().await {
    // …
}

pub fn next_n_rows( &mut self, max_rows: usize, max_bytes: usize, ) -> NextNRowsOutput

Append up to max_rows rows of data into freshly allocated per-column arrays.

Throughput-oriented sibling of Self::next_row: shares the streaming-join machinery but amortizes per-row allocation by batching MutableArrayData extends and only finalizing to ArrayRef once per call.

The returned NextNRowsOutput::columns strictly follows the schema specified by Self::schema, with num_rows == 0 signalling exhaustion.

max_bytes caps the estimated output-buffer footprint of the batch (sum of per-row source-array bytes, computed from ArrayData::get_array_memory_size / len). Pass usize::MAX to disable the byte cap. The first row is always admitted regardless of the cap, so a single wide row can exceed max_bytes.

pub fn next_n_rows_async( &mut self, max_rows: usize, max_bytes: usize, ) -> impl Future<Output = NextNRowsOutput> + use<'_, E>

where E: 'static + Send + Clone,

Asynchronous sibling of Self::next_n_rows.

pub fn next_row_batch(&mut self) -> Option<ArrowRecordBatch>

Calls Self::next_row and wraps the result in a ArrowRecordBatch.

Only use this if you absolutely need a ArrowRecordBatch as this adds a some overhead for schema validation.

See Self::next_row for more information.

pub async fn next_row_batch_async(&mut self) -> Option<ArrowRecordBatch>

where E: 'static + Send + Clone,

impl<E: StorageEngineLike> QueryHandle<E>

pub fn iter(&mut self) -> impl Iterator<Item = Vec<ArrowArrayRef>> + '_

Returns an iterator backed by Self::next_row.

pub fn into_iter(self) -> impl Iterator<Item = Vec<ArrowArrayRef>>

Returns an iterator backed by Self::next_row.

pub fn batch_iter(&mut self) -> impl Iterator<Item = ArrowRecordBatch> + '_

Returns an iterator backed by Self::next_row_batch.

pub fn into_batch_iter(self) -> impl Iterator<Item = ArrowRecordBatch>

Returns an iterator backed by Self::next_row_batch.