pub enum WriterStatistics {
None,
MinMax,
Automatic,
}Expand description
The block-local min/max statistics policy selected by a crate::Writer.
Statistics are written only for v0.2 logical types with a fixed canonical
representation: booleans, numeric types, decimals, timestamps, dates, and
fixed-width binary values. Variable-width strings and binary values have no
v0.2 min/max representation and are left without statistics under either
enabled policy. None is the default and preserves the writer’s historical
byte output.
Under both enabled policies nulls and floating-point NaNs are ignored, a
column left with no value has no statistic at all, and infinities are
ordinary bounds. Floating-point bounds are numeric, so -0.0 and 0.0 are
interchangeable as a bound; the writer keeps whichever bit pattern it saw
first, which makes the choice deterministic without claiming that one
encoding of zero is smaller than the other.
A pair costs twice its logical type’s canonical width and lives in the data
frame header, which readers bound at 64 MiB. A fixed_binary column
therefore charges twice its byte width against that budget, and a schema
wide enough to exhaust it is refused by crate::Writer::append rather
than written; such a schema can still be written with Self::None.
Variants§
None
Do not write optional column statistics.
MinMax
Write min/max statistics whenever a supported column has at least one non-null, non-NaN value.
This includes the primary timestamp column. Section 11 makes that
repetition unnecessary, because the block header already carries the
same bounds as the mandatory pruning statistic, but writing it anyway
keeps this policy’s output a function of the schema alone and is what
lets it override an Self::Automatic omission on any column.
Automatic
Write statistics only where a deterministic rule predicts that the header bytes are justified.
The rule is exactly this. A column gets a pair when it is not the
primary timestamp column, whose block-header bounds are already the
complete mandatory pruning statistic; and it has at least 64 non-null,
non-NaN values; and its raw dense value bytes are at least eight times
the pair. The last test binds only on bool, whose values cost one bit
each, where it raises the effective floor to 121 values; for every other
supported type the 64-value floor is the stricter of the two.
The decision reads only the block’s values and its schema. It does not
depend on elapsed time, iteration order, randomness, earlier blocks, or
how the rows were divided across crate::Writer::append calls, so
equal block contents produce equal bytes.
Trait Implementations§
Source§impl Clone for WriterStatistics
impl Clone for WriterStatistics
Source§fn clone(&self) -> WriterStatistics
fn clone(&self) -> WriterStatistics
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source. Read more