Module pb 

Protobuf definitions for Lance Format

Modules

data_fragment

Nested message and enum types in DataFragment.

data_overlay_file

Nested message and enum types in DataOverlayFile.

deletion_file

Nested message and enum types in DeletionFile.

encoded_u64_array

Nested message and enum types in EncodedU64Array.

fragment_action

Nested message and enum types in FragmentAction.

fragment_reuse_index_details

Nested message and enum types in FragmentReuseIndexDetails.

fragment_tree

Nested message and enum types in FragmentTree.

fragment_tree_metadata

Nested message and enum types in FragmentTreeMetadata.

manifest

Nested message and enum types in Manifest.

transaction

Nested message and enum types in Transaction.

u64_segment

Nested message and enum types in U64Segment.

Structs

AddDataFile

Append to the ordered file list.

AddDeletionFile

BasePath

external dataset base path

ClearDeletionFile

CompactedSsTable

A pointer to the latest SSTable compacted for a shard.

DataFile

DataFragment

A DataFragment is a set of files which represent the different columns of the same rows. If column exists in the schema of a dataset, but the file for that column does not exist within a DataFragment of that dataset, that column consists entirely of nulls.

DataOverlayFile

An overlay file supplies new values for a subset of (row offset, field) cells within a fragment, without rewriting the fragment’s base data files. It is used for efficient updates when only a small fraction of rows and/or columns change.

DeletionFile

Deletion File

EncodedU64Array

/ A basic bitpacked array of u64 values.

ExternalFile

A byte range of a file. Used by the fragment reuse index details.

FieldCoverage

Per-field coverage for a sparse overlay.

FragmentAction

upsert_fragment replaces the complete record. Use it for changes that other actions cannot express, including overlays, version sequences, and row IDs.

FragmentReuseIndexDetails

FragmentTree

Fragment state for one Version Manifest.

FragmentTreeChild

Children are ordered by min_key. A child owns [min_key, next min_key); the last child inherits its parent’s upper bound.

FragmentTreeMetadata

Fragment storage layout for this version.

FragmentTreeMutation

Validated mutation with exact after-minus-before aggregate deltas.

FragmentTreeNode

Interior routing node.

FragmentTreeRoot

Inline or external root state.

IndexCatchupProgress

Tracks which compacted SSTable generation a base table index has been rebuilt to cover. Used to determine whether to read from SSTable indexes or base table.

IndexFile

Metadata about a single file within an index segment.

IndexMetadata

Metadata describing an index.

IndexSection

Index Section, containing a list of index metadata for one dataset version.

Manifest

Manifest is a global section shared between all the files.

MemWalIndexDetails

Index details for MemWAL Index, stored in IndexMetadata.index_details. This is the centralized structure for all MemWAL metadata:

RemoveDataFile

Remove every matching path, preserving order.

ReplaceDataFile

On the first expected_path match, replace path, file_size_bytes, and base_id. Preserve its column mapping and file version.

RowDatasetVersionRun

/ A run of rows with the same version.

RowDatasetVersionSequence

A sequence of dataset versions. Similar to RowIdSequence but tracks version runs. It uses RLE (Run-Length Encoding) to efficiently represent consecutive rows with the same version.

RowIdSequence

A sequence of row IDs. This is split up into one or more segments, each of which can be encoded in different ways. The encodings are optimized for values that are sorted, which will often be the case with row ids. They also have optimized forms depending on how sparse the values are.

RowLineageColumn

Marks a row lineage sequence that is stored as a hidden column of one of the fragment’s files rather than inline: _rowid under the reserved field id -3, _row_created_at_version under -4, _row_last_updated_at_version under -5. Exactly one entry of DataFragment.files carries that id in its fields, and its column_indices locates the column the way it does for a user column, so the column may share a file with the user columns or with the other lineage columns, or sit in a file that holds nothing else. The column is a non-nullable uint64 with exactly physical_rows values, in physical row order; see the table format documentation for the schema.

ShardFieldEntry

A shard field value stored as raw Arrow scalar bytes.

ShardManifest

Shard manifest containing epoch-based fencing and WAL state. Each shard has exactly one active writer at any time.

ShardingField

Sharding field definition.

ShardingSpec

Sharding spec definition.

SsTable

An SSTable: the immutable result of flushing a MemTable, stored as a Lance dataset.

Transaction

A transaction represents the changes to a dataset.

U64Segment

/ Different ways to encode a sequence of u64 values.

Uuid

UUID type. encoded as 16 bytes.

VersionAuxData

Auxiliary Data attached to a version. Only load on-demand.

Enums

ShardStatus

Lifecycle status of a WAL shard. Drives drop-table two-phase commit: a SEALED shard refuses new writer claims (reversible) until the drop commits (the shard dir is deleted) or rolls back (status -> ACTIVE).