Module transaction 

Transaction definitions for updating datasets

Prior to creating a new manifest, a transaction must be created representing the changes being made to the dataset. By representing them as incremental changes, we can detect whether concurrent operations are compatible with one another. We can also rebuild manifests when retrying committing a manifest.

For more details please refer to the Transaction Specification.

The work splits along these lines:

builder            Transaction: an operation plus the version it was based on
operation          the vocabulary of changes an operation can describe
update_map         incremental edits to the manifest's string maps
validate           pre-commit checks against the manifest being replaced
manifest_build     applying an operation to produce the next manifest
index_maintenance  how that narrows or drops index metadata
row_version        how it assigns row ids and per-row version metadata
conflicts          whether two operations collide, for the commit retry path
proto              the persisted protobuf encoding of all of the above

Structs

CoverageIdentity

What one physical index segment contributes to coverage.

DataOverlayGroup

Overlay files to append to a single fragment, in order (the last entry is newest). The overlays are appended to the fragment’s existing overlays list rather than replacing it, so overlays written by concurrent commits are preserved. Each overlay’s committed_version is stamped to the new dataset version at commit time (re-stamped on retry).

DataReplacementGroup

ReadVersionState

The version a transaction read, as the coverage derivation needs it.

RewriteGroup

RewrittenIndex

Transaction

A change to a dataset that can be retried

TransactionBuilder

Add TransactionBuilder for flexibly setting option without using mut

UpdateMap

Represents updates to a map (either incremental or replacement)

UpdateMapEntry

An entry for a map update. If value is None, the key will be removed from the map.

UpdatedFragmentOffsets

Matched physical row offsets per fragment for a partial UpdateMode::RewriteColumns update.

Enums

Operation

An operation on a dataset.

UpdateMode

Functions

translate_config_updates

Helper function to translate old-style config updates to new UpdateMap format

translate_schema_metadata_updates

Helper function to translate old-style schema metadata to new UpdateMap format

validate_operation

Validate the operation is valid for the given manifest.

Type Aliases

LogicalIndexSegments

Non-system logical index name -> its physical segments, ordered by UUID.