Expand description
§SuperTable Manifest and Snapshot Management
This module defines the structures for tracking table state and file inventory. The design follows Apache Iceberg’s snapshot-based model, where each snapshot represents an immutable, point-in-time view of the table.
§Key Concepts
- Snapshot: A complete view of the table at a point in time
- Manifest List: A file listing all manifest files for a snapshot
- Manifest File: A file listing data files with their metadata
- Data File: An actual Parquet file containing table data
§Immutability
Once committed, snapshots are immutable. Updates create new snapshots that reference both new and existing data files. This enables:
- Time travel (querying historical states)
- Atomic commits (all-or-nothing updates)
- Concurrent reads (readers see consistent snapshots)
Structs§
- Data
File - Metadata about a data or delete file.
- Manifest
Entry - An entry in a manifest file, representing a single data file.
- Manifest
File - A manifest file contains a list of data files.
- Manifest
File Entry - An entry in a manifest list, referencing a single manifest file.
- Manifest
List - A manifest list contains references to all manifest files for a snapshot.
- Snapshot
- A snapshot represents an immutable view of the table at a point in time.
- Snapshot
Builder - Builder for constructing
Snapshotinstances.
Enums§
- File
Content - Type of content stored in a file.
- File
Format - Supported data file formats.
- Manifest
Content - The type of content in a manifest file.
- Manifest
Entry Status - The status of a manifest entry.
- Operation
- The type of operation that created a snapshot.