pub struct ColumnMeta {
pub name: String,
pub data_type: u8,
pub role: u8,
pub default_encoding: u8,
pub stats: ColumnStats,
pub bloom_filter: Option<Vec<u8>>,
pub encrypted: bool,
pub key_id: Option<String>,
pub decimal_scale: Option<u8>,
pub row_group_blooms: Option<Vec<Vec<u8>>>,
}Expand description
Metadata for a single column within a segment.
§Segment-level statistics
The stats field stores aggregate statistics across all
row groups: min_value, max_value, null_count, value_count,
sum, and distinct_count. These are computed by the segment writer
during encoding and merged via merge_stats(). Query-time code can
use these to prune entire segments without reading row-group data.
Metadata is kept proportional to the data: no fixed-size sketches or
histograms, which cost ~49 KiB per segment however little it holds, and
which the planner does not need — distinct_count above is exact.
Fields§
§name: StringColumn name.
data_type: u8Column data type tag (maps to ColumnType).
role: u8Column role tag: 0 = timestamp, 1 = tag, 2 = field.
default_encoding: u8The encoding the writer intended for this column, chosen from the column’s type before any data was seen.
This is a hint only. The adaptive selector may fall back to a different
codec per block (e.g. Chimp → Gorilla → plain), so the authoritative
encoding for reading is always
ColumnBlockMeta::encoding.
The field was previously named encoding, which read as a statement of
fact; nothing decodes from it, but the name invited that mistake.
stats: ColumnStatsGlobal statistics across all row groups (min, max, null_count, value_count, sum, distinct_count). Enables segment-level pruning at query time.
bloom_filter: Option<Vec<u8>>Bloom filter for tag columns.
Contains the serialized bloom filter bit array for tag columns,
enabling segment-level predicate pushdown: if a queried tag value
is definitely not in the bloom filter, the segment can be skipped.
None for non-tag columns, and for segments written before per-tag
row-group blooms existed.
encrypted: boolWhether this column is encrypted at rest (field-level encryption).
When true, all column blocks for this column are AES-256-GCM
encrypted and key_id identifies the encryption key.
Statistics (min/max/sum/distinct) are zeroed to prevent leakage.
key_id: Option<String>Encryption key ID for this column. Used by the reader to look up
the decryption key from a FieldKeyProvider.
decimal_scale: Option<u8>Digits after the decimal point, for a decimal column.
None for every other type. The scale lives here, once per column,
rather than beside each value: it is a property of the column, fixed
when the column was created, and the write path rescales every value
to it before the memtable ever sees it. Storing it per value would
cost a byte a row to say the same thing 65 536 times, and would make
a segment whose rows disagreed representable.
row_group_blooms: Option<Vec<Vec<u8>>>Per-row-group bloom filters for string columns .
row_group_blooms[rg_idx] is a serialized bloom filter containing
all distinct string values for this column in that row group.
Enables row-group-level predicate pushdown: if a queried value is
definitely not in the bloom filter, the row group can be skipped.
None for non-string columns, encrypted columns, or segments
written before .
Implementations§
Source§impl ColumnMeta
impl ColumnMeta
Sourcepub fn arrow_data_type(&self) -> DataType
pub fn arrow_data_type(&self) -> DataType
The Arrow type this column decodes to.
The single answer to a question four places in the reader used to
answer separately, each ending in _ => DataType::Utf8. That
fallback was fine while every non-numeric column was a string; it
silently turned a decimal column into Utf8 the moment one existed.
Sourcepub fn empty_array(&self) -> ArrayRef
pub fn empty_array(&self) -> ArrayRef
An empty array of this column’s Arrow type, for a pruned row group.
Sourcepub fn arrow_field(&self) -> Field
pub fn arrow_field(&self) -> Field
The Arrow field for this column, carrying its role as metadata.
Trait Implementations§
Source§impl Clone for ColumnMeta
impl Clone for ColumnMeta
Source§fn clone(&self) -> ColumnMeta
fn clone(&self) -> ColumnMeta
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source. Read moreSource§impl Debug for ColumnMeta
impl Debug for ColumnMeta
Source§impl<'de> Deserialize<'de> for ColumnMeta
impl<'de> Deserialize<'de> for ColumnMeta
Source§fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
Source§impl PartialEq for ColumnMeta
impl PartialEq for ColumnMeta
Source§impl Serialize for ColumnMeta
impl Serialize for ColumnMeta
impl StructuralPartialEq for ColumnMeta
Auto Trait Implementations§
impl Freeze for ColumnMeta
impl RefUnwindSafe for ColumnMeta
impl Send for ColumnMeta
impl Sync for ColumnMeta
impl Unpin for ColumnMeta
impl UnsafeUnpin for ColumnMeta
impl UnwindSafe for ColumnMeta
Blanket Implementations§
impl<T> Allocation for T
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> DeserializeOwned for Twhere
T: for<'de> Deserialize<'de>,
Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§fn in_current_span(self) -> Instrumented<Self> ⓘ
fn in_current_span(self) -> Instrumented<Self> ⓘ
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more