Struct BTreeMemIndex 

pub struct BTreeMemIndex { /* private fields */ }

In-memory BTree index for scalar fields.

Represents the in-memory portion of Lance’s on-disk BTree index. Implemented using crossbeam-skiplist for concurrent access with O(log n) operations.

Implementations

impl BTreeMemIndex

pub fn new(field_id: i32, column_name: String) -> Self

Create a new BTree index for the given field.

pub fn field_id(&self) -> i32

Get the field ID this index is built on.

pub fn insert(&self, batch: &RecordBatch, row_offset: u64) -> Result<()>

Insert rows from a batch into the index.

pub fn get(&self, value: &ScalarValue) -> Vec<u64>

Look up row positions for an exact value.

pub fn len(&self) -> usize

Get the number of entries (not unique values).

pub fn is_empty(&self) -> bool

Check if the index is empty.

pub fn column_name(&self) -> &str

Get the column name.

pub fn snapshot(&self) -> Vec<(OrderableScalarValue, Vec<u64>)>

Get a snapshot of all entries grouped by value in sorted order.

pub fn data_type(&self) -> Option<DataType>

Get the data type of the indexed column.

Returns None if the index is empty.

pub fn to_training_batches(&self, batch_size: usize) -> Result<Vec<RecordBatch>>

Export the index data as sorted RecordBatches for BTree index training.

pub fn to_training_batches_reversed( &self, batch_size: usize, total_rows: usize, ) -> Result<Vec<RecordBatch>>

Export the index data as sorted RecordBatches with reversed row positions.

This is used when flushing MemTable to disk with batches in reverse order. Since the flushed data will have rows in reverse order, we need to map the row positions accordingly: reversed_position = total_rows - original_position - 1

Arguments

• batch_size - Maximum number of entries per batch
• total_rows - Total number of rows in the MemTable (needed for position reversal)

Trait Implementations

impl Debug for BTreeMemIndex

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.