Expand description
Manifest file handling and data file tracking for Iceberg tables.
This module provides the core types and implementations for working with manifest files, which track the data files that comprise an Iceberg table. Key components include:
ManifestEntry- Entries tracking data file additions and deletionsDataFile- Metadata about data files including statistics and metricsContent- Types of content stored in data files (data vs deletes)Status- Tracking additions and deletions of filesFileFormat- Supported file formats (Avro, ORC, Parquet)
Manifest files are a key part of Iceberg’s architecture, providing metadata about data files and enabling efficient file pruning during queries.
Structs§
- AvroMap
- Utility struct to convert avro maps to rust hashmaps. Derefences to a Hashmap.
- Data
File - DataFile found in Manifest.
- Data
File Builder - Builder for
DataFile. - Data
File V1 - DataFile found in Manifest.
- Data
File V2 - DataFile found in Manifest.
- Manifest
Entry - Entry in manifest with the iceberg spec version 2.
- Manifest
Entry Builder - Builder for
ManifestEntry. - Manifest
Entry V1 - Entry in manifest with the iceberg spec version 1.
- Manifest
Entry V2 - Entry in manifest with the iceberg spec version 2.
Enums§
- Content
- Type of content stored by the data file.
- File
Format - Name of file format
- Manifest
Entry Enum - Entry in manifest
- Status
- Used to track additions and deletions
Functions§
- partition_
value_ schema - Get schema for partition values depending on partition spec and table schema