Module pb 

Protobuf definitions for Lance Format

Modules

data_fragment

Nested message and enum types in DataFragment.

deletion_file

Nested message and enum types in DeletionFile.

encoded_u64_array

Nested message and enum types in EncodedU64Array.

fragment_reuse_index_details

Nested message and enum types in FragmentReuseIndexDetails.

manifest

Nested message and enum types in Manifest.

transaction

Nested message and enum types in Transaction.

u64_segment

Nested message and enum types in U64Segment.

Structs

BasePath

external dataset base path

DataFile

DataFragment

A DataFragment is a set of files which represent the different columns of the same rows. If column exists in the schema of a dataset, but the file for that column does not exist within a DataFragment of that dataset, that column consists entirely of nulls.

DeletionFile

Deletion File

EncodedU64Array

/ A basic bitpacked array of u64 values.

ExternalFile

FlushedGeneration

A flushed MemTable generation and its storage location.

FragmentReuseIndexDetails

IndexCatchupProgress

Tracks which merged generation a base table index has been rebuilt to cover. Used to determine whether to read from flushed MemTable indexes or base table.

IndexFile

Metadata about a single file within an index segment.

IndexMetadata

Metadata describing an index.

IndexSection

Index Section, containing a list of index metadata for one dataset version.

Manifest

Manifest is a global section shared between all the files.

MemWalIndexDetails

Index details for MemWAL Index, stored in IndexMetadata.index_details. This is the centralized structure for all MemWAL metadata:

MergedGeneration

A shard’s merged generation, used in MemWalIndexDetails.

RowDatasetVersionRun

/ A run of rows with the same version.

RowDatasetVersionSequence

/ A sequence of dataset versions. Similar to RowIdSequence but tracks / version runs. It uses RLE (Run-Length Encoding) to efficiently represent consecutive rows with the same version.

RowIdSequence

/ A sequence of row IDs. This is split up into one or more segments, / each of which can be encoded in different ways. The encodings are optimized / for values that are sorted, which will often be the case with row ids. / They also have optimized forms depending on how sparse the values are.

ShardFieldEntry

A shard field value stored as raw Arrow scalar bytes.

ShardManifest

Shard manifest containing epoch-based fencing and WAL state. Each shard has exactly one active writer at any time.

ShardingField

Sharding field definition.

ShardingSpec

Sharding spec definition.

Transaction

A transaction represents the changes to a dataset.

U64Segment

/ Different ways to encode a sequence of u64 values.

Uuid

UUID type. encoded as 16 bytes.

VersionAuxData

Auxiliary Data attached to a version. Only load on-demand.