Module pb 

Protobuf definitions for Lance Format

Modules

data_fragment

Nested message and enum types in DataFragment.

deletion_file

Nested message and enum types in DeletionFile.

encoded_u64_array

Nested message and enum types in EncodedU64Array.

fragment_reuse_index_details

Nested message and enum types in FragmentReuseIndexDetails.

manifest

Nested message and enum types in Manifest.

transaction

Nested message and enum types in Transaction.

u64_segment

Nested message and enum types in U64Segment.

Structs

BasePath

external dataset base path

DataFile

DataFragment

A DataFragment is a set of files which represent the different columns of the same rows. If column exists in the schema of a dataset, but the file for that column does not exist within a DataFragment of that dataset, that column consists entirely of nulls.

DeletionFile

Deletion File

EncodedU64Array

/ A basic bitpacked array of u64 values.

ExternalFile

FlushedGeneration

A flushed MemTable generation and its storage location.

FragmentReuseIndexDetails

IndexCatchupProgress

Tracks which merged generation a base table index has been rebuilt to cover. Used to determine whether to read from flushed MemTable indexes or base table.

IndexMetadata

Metadata describing an index.

IndexSection

Index Section, containing a list of index metadata for one dataset version.

Manifest

Manifest is a global section shared between all the files.

MemWalIndexDetails

Index details for MemWAL Index, stored in IndexMetadata.index_details. This is the centralized structure for all MemWAL metadata:

MergedGeneration

A region’s merged generation, used in MemWalIndexDetails.

RegionField

Region field definition.

RegionManifest

Region manifest containing epoch-based fencing and WAL state. Each region has exactly one active writer at any time.

RegionSpec

Region spec definition.

RowDatasetVersionRun

/ A run of rows with the same version.

RowDatasetVersionSequence

/ A sequence of dataset versions. Similar to RowIdSequence but tracks / version runs. It uses RLE (Run-Length Encoding) to efficiently represent consecutive rows with the same version.

RowIdSequence

/ A sequence of row IDs. This is split up into one or more segments, / each of which can be encoded in different ways. The encodings are optimized / for values that are sorted, which will often be the case with row ids. / They also have optimized forms depending on how sparse the values are.

Transaction

A transaction represents the changes to a dataset.

U64Segment

/ Different ways to encode a sequence of u64 values.

Uuid

UUID type. encoded as 16 bytes.

VectorIndexDetails

Empty details messages for older indexes that don’t take advantage of the details field.

VersionAuxData

Auxiliary Data attached to a version. Only load on-demand.