Struct deltalake::datafusion::parquet::format::ColumnIndex

pub struct ColumnIndex {
    pub null_pages: Vec<bool>,
    pub min_values: Vec<Vec<u8>>,
    pub max_values: Vec<Vec<u8>>,
    pub boundary_order: BoundaryOrder,
    pub null_counts: Option<Vec<i64>>,
}

Description for ColumnIndex. Each <array-field>[i] refers to the page at OffsetIndex.page_locations[i]

Fields

null_pages: Vec<bool>

A list of Boolean values to determine the validity of the corresponding min and max values. If true, a page contains only null values, and writers have to set the corresponding entries in min_values and max_values to byte[0], so that all lists have the same length. If false, the corresponding entries in min_values and max_values must be valid.

min_values: Vec<Vec<u8>>

Two lists containing lower and upper bounds for the values of each page determined by the ColumnOrder of the column. These may be the actual minimum and maximum values found on a page, but can also be (more compact) values that do not exist on a page. For example, instead of storing ““Blart Versenwald III”, a writer may set min_values[i]=“B”, max_values[i]=“C”. Such more compact values must still be valid values within the column’s logical type. Readers must make sure that list entries are populated before using them by inspecting null_pages.

max_values: Vec<Vec<u8>>

boundary_order: BoundaryOrder

Stores whether both min_values and max_values are ordered and if so, in which direction. This allows readers to perform binary searches in both lists. Readers cannot assume that max_values[i] <= min_values[i+1], even if the lists are ordered.

null_counts: Option<Vec<i64>>

A list containing the number of null values for each page *

Implementations

impl ColumnIndex

pub fn new<F5>( null_pages: Vec<bool>, min_values: Vec<Vec<u8>>, max_values: Vec<Vec<u8>>, boundary_order: BoundaryOrder, null_counts: F5 ) -> ColumnIndex

where F5: Into<Option<Vec<i64>>>,

Trait Implementations

impl Clone for ColumnIndex

fn clone(&self) -> ColumnIndex

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ColumnIndex

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl Hash for ColumnIndex

fn hash<__H>(&self, state: &mut __H)

where __H: Hasher,

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl Ord for ColumnIndex

fn cmp(&self, other: &ColumnIndex) -> Ordering

This method returns an Ordering between self and other.

fn max(self, other: Self) -> Self

where Self: Sized,

Compares and returns the maximum of two values.

fn min(self, other: Self) -> Self

where Self: Sized,

Compares and returns the minimum of two values.

fn clamp(self, min: Self, max: Self) -> Self

where Self: Sized + PartialOrd,

Restrict a value to a certain interval.

impl PartialEq for ColumnIndex

fn eq(&self, other: &ColumnIndex) -> bool

This method tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

This method tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl PartialOrd for ColumnIndex

fn partial_cmp(&self, other: &ColumnIndex) -> Option<Ordering>

This method returns an ordering between self and other values if one exists.

fn lt(&self, other: &Rhs) -> bool

This method tests less than (for self and other) and is used by the < operator.

fn le(&self, other: &Rhs) -> bool

This method tests less than or equal to (for self and other) and is used by the <= operator.

fn gt(&self, other: &Rhs) -> bool

This method tests greater than (for self and other) and is used by the > operator.

fn ge(&self, other: &Rhs) -> bool

This method tests greater than or equal to (for self and other) and is used by the >= operator.

impl TSerializable for ColumnIndex

fn read_from_in_protocol<T>(i_prot: &mut T) -> Result<ColumnIndex, Error>

where T: TInputProtocol,

fn write_to_out_protocol<T>(&self, o_prot: &mut T) -> Result<(), Error>

where T: TOutputProtocol,

impl Eq for ColumnIndex

impl StructuralPartialEq for ColumnIndex