Struct PartitionPruningStatistics 

pub struct PartitionPruningStatistics { /* private fields */ }

👎Deprecated since 52.0.0:

This struct is no longer used internally. Use replace_columns_with_literals from datafusion-physical-expr-adapter to substitute partition column values before pruning. It will be removed in 58.0.0 or 6 months after 52.0.0 is released, whichever comes first.

Prune files based on their partition values.

This is used both at planning time and execution time to prune files based on their partition values. This feeds into CompositePruningStatistics to allow pruning with filters that depend both on partition columns and data columns (e.g. WHERE partition_col = data_col).

Implementations

impl PartitionPruningStatistics

pub fn try_new( partition_values: Vec<Vec<ScalarValue>>, partition_fields: Vec<Arc<Field>>, ) -> Result<PartitionPruningStatistics, DataFusionError>

Create a new instance of PartitionPruningStatistics.

Args:

• partition_values: A vector of vectors of ScalarValues. The outer vector represents the containers while the inner vector represents the partition values for each column. Note that this is the opposite of the order of the partition columns in PartitionPruningStatistics::partition_schema.
• partition_schema: The schema of the partition columns. This must not be the schema of the entire file or table: instead it must only be the schema of the partition columns, in the same order as the values in partition_values.

Example

To create PartitionPruningStatistics for two partition columns a and b, for three containers like this:

a	b
1	2
3	4
5	6

let partition_values = vec![
    vec![ScalarValue::from(1i32), ScalarValue::from(2i32)],
    vec![ScalarValue::from(3i32), ScalarValue::from(4i32)],
    vec![ScalarValue::from(5i32), ScalarValue::from(6i32)],
];
let partition_fields = vec![
    Arc::new(Field::new("a", DataType::Int32, false)),
    Arc::new(Field::new("b", DataType::Int32, false)),
];
let partition_stats =
    PartitionPruningStatistics::try_new(partition_values, partition_fields).unwrap();

Trait Implementations

impl Clone for PartitionPruningStatistics

fn clone(&self) -> PartitionPruningStatistics

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl PruningStatistics for PartitionPruningStatistics

fn min_values(&self, column: &Column) -> Option<Arc<dyn Array>>

Return the minimum values for the named column, if known.

fn max_values(&self, column: &Column) -> Option<Arc<dyn Array>>

Return the maximum values for the named column, if known.

fn num_containers(&self) -> usize

Return the number of containers (e.g. Row Groups) being pruned with these statistics.

fn null_counts(&self, _column: &Column) -> Option<Arc<dyn Array>>

Return the number of null values for the named column as an UInt64Array

fn row_counts(&self) -> Option<Arc<dyn Array>>

Return the number of rows in each container as an UInt64Array.

fn contained( &self, column: &Column, values: &HashSet<ScalarValue>, ) -> Option<BooleanArray>

Returns BooleanArray where each row represents information known about specific literal values in a column.