Struct Statistics

pub struct Statistics {
    pub num_rows: Precision<usize>,
    pub total_byte_size: Precision<usize>,
    pub column_statistics: Vec<ColumnStatistics>,
}

Statistics for a relation Fields are optional and can be inexact because the sources sometimes provide approximate estimates for performance reasons and the transformations output are not always predictable.

Fields

num_rows: Precision<usize>

The number of table rows.

total_byte_size: Precision<usize>

Total bytes of the table rows.

column_statistics: Vec<ColumnStatistics>

Statistics on a column level.

It must contains a ColumnStatistics for each field in the schema of the table to which the Statistics refer.

Implementations

impl Statistics

pub fn new_unknown(schema: &Schema) -> Statistics

Returns a Statistics instance for the given schema by assigning unknown statistics to each column in the schema.

pub fn unknown_column(schema: &Schema) -> Vec<ColumnStatistics>

Returns an unbounded ColumnStatistics for each field in the schema.

pub fn with_num_rows(self, num_rows: Precision<usize>) -> Statistics

Set the number of rows

pub fn with_total_byte_size( self, total_byte_size: Precision<usize>, ) -> Statistics

Set the total size, in bytes

pub fn add_column_statistics(self, column_stats: ColumnStatistics) -> Statistics

Add a column to the column statistics

pub fn to_inexact(self) -> Statistics

If the exactness of a Statistics instance is lost, this function relaxes the exactness of all information by converting them Precision::Inexact.

pub fn project(self, projection: Option<&Vec<usize>>) -> Statistics

Project the statistics to the given column indices.

For example, if we had statistics for columns {"a", "b", "c"}, projecting to vec![2, 1] would return statistics for columns {"c", "b"}.

pub fn with_fetch( self, schema: Arc<Schema>, fetch: Option<usize>, skip: usize, n_partitions: usize, ) -> Result<Statistics, DataFusionError>

Calculates the statistics after applying fetch and skip operations.

Here, self denotes per-partition statistics. Use the n_partitions parameter to compute global statistics in a multi-partition setting.

pub fn try_merge_iter<'a, I>( items: I, schema: &Schema, ) -> Result<Statistics, DataFusionError>

where I: IntoIterator<Item = &'a Statistics>,

Summarize zero or more statistics into a single Statistics instance.

Returns an error if the statistics do not match the specified schemas.

pub fn try_merge( self, other: &Statistics, ) -> Result<Statistics, DataFusionError>

Merge this Statistics value with another Statistics value.

Returns an error if the statistics do not match (different schemas).

Example

let stats1 = Statistics::default()
  .with_num_rows(Precision::Exact(1))
  .with_total_byte_size(Precision::Exact(2))
  .add_column_statistics(ColumnStatistics::new_unknown()
     .with_null_count(Precision::Exact(3))
     .with_min_value(Precision::Exact(ScalarValue::from(4)))
     .with_max_value(Precision::Exact(ScalarValue::from(5)))
  );

let stats2 = Statistics::default()
  .with_num_rows(Precision::Exact(10))
  .with_total_byte_size(Precision::Inexact(20))
  .add_column_statistics(ColumnStatistics::new_unknown()
      // absent null count
     .with_min_value(Precision::Exact(ScalarValue::from(40)))
     .with_max_value(Precision::Exact(ScalarValue::from(50)))
  );

let merged_stats = stats1.try_merge(&stats2).unwrap();
let expected_stats = Statistics::default()
  .with_num_rows(Precision::Exact(11))
  .with_total_byte_size(Precision::Inexact(22)) // inexact in stats2 --> inexact
  .add_column_statistics(
    ColumnStatistics::new_unknown()
      .with_null_count(Precision::Absent) // missing from stats2 --> absent
      .with_min_value(Precision::Exact(ScalarValue::from(4)))
      .with_max_value(Precision::Exact(ScalarValue::from(50)))
  );

assert_eq!(merged_stats, expected_stats)

Trait Implementations

impl Clone for Statistics

fn clone(&self) -> Statistics

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Statistics

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl Default for Statistics

fn default() -> Statistics

Returns a new Statistics instance with all fields set to unknown and no columns.

impl Display for Statistics

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl PartialEq for Statistics

fn eq(&self, other: &Statistics) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl TryFrom<&Statistics> for Statistics

type Error = DataFusionError

The type returned in the event of a conversion error.

fn try_from( s: &Statistics, ) -> Result<Statistics, <Statistics as TryFrom<&Statistics>>::Error>

Performs the conversion.

impl Eq for Statistics

impl StructuralPartialEq for Statistics