Struct lance_encoding::encodings::logical::struct::SimpleStructScheduler

pub struct SimpleStructScheduler { /* private fields */ }

A scheduler for structs

The implementation is actually a bit more tricky than one might initially think. We can’t just go through and schedule each column one after the other. This would mean our decode can’t start until nearly all the data has arrived (since we need data from each column)

Instead, we schedule in row-major fashion, described in detail below.

Note: this scheduler is the starting point for all decoding. This is because we treat the top-level record batch as a non-nullable struct.

This means this scheduler has to deal with u64 indices / ranges because the top-level scheduler spans up to u64 rows.

Implementations

impl SimpleStructScheduler

pub fn new( children: Vec<Vec<Arc<dyn LogicalPageScheduler>>>, child_fields: Fields ) -> Self

pub fn new_root( children: Vec<Vec<Arc<dyn LogicalPageScheduler>>>, child_fields: Fields ) -> Self

pub fn new_root_decoder_ranges( &self, ranges: &[Range<u64>] ) -> SimpleStructDecoder

pub fn new_root_decoder_indices(&self, indices: &[u64]) -> SimpleStructDecoder

pub fn schedule_ranges_u64( &self, ranges: &[Range<u64>], context: &mut SchedulerContext, top_level_row: u64 ) -> Result<()>

pub fn schedule_take_u64( &self, indices: &[u64], context: &mut SchedulerContext, top_level_row: u64 ) -> Result<()>

Trait Implementations

impl Debug for SimpleStructScheduler

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl LogicalPageScheduler for SimpleStructScheduler

fn schedule_ranges( &self, ranges: &[Range<u32>], context: &mut SchedulerContext, top_level_row: u64 ) -> Result<()>

Schedules I/O for the requested portions of the page.

fn num_rows(&self) -> u32

The number of rows covered by this page

fn schedule_take( &self, indices: &[u32], context: &mut SchedulerContext, top_level_row: u64 ) -> Result<()>

Schedules I/O for the requested rows (identified by row offsets from start of page) TODO: implement this using schedule_ranges