Struct MiniBlockChunk 

pub struct MiniBlockChunk {
    pub buffer_sizes: Vec<u32>,
    pub log_num_values: u8,
}

Describes the size of a mini-block chunk of data

Mini-block chunks are designed to be small (just a few disk sectors) and contain a power-of-two number of values (except for the last chunk)

By default we limit a chunk to 4Ki values and slightly less than 8KiB of compressed value data. The byte budget remains the primary constraint, so only encodings that compress many values into that budget can use larger value counts when explicitly configured.

The maximum number of values per chunk can be configured via the LANCE_MINIBLOCK_MAX_VALUES environment variable. This is only useful in extremely bandwidth-limited environments; the default is appropriate for local disks and same-region cloud object storage.

Fields

buffer_sizes: Vec<u32>

log_num_values: u8

Implementations

impl MiniBlockChunk

pub fn num_values(&self, vals_in_prev_blocks: u64, total_num_values: u64) -> u64

Gets the number of values in this block

This requires vals_in_prev_blocks and total_num_values because the last block in a page is a special case which stores 0 for log_num_values and, in that case, the number of values is determined by subtracting vals_in_prev_blocks from total_num_values

Trait Implementations

impl Debug for MiniBlockChunk

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.