Module transaction 

Transaction definitions for updating datasets

Prior to creating a new manifest, a transaction must be created representing the changes being made to the dataset. By representing them as incremental changes, we can detect whether concurrent operations are compatible with one another. We can also rebuild manifests when retrying committing a manifest.

For more details please refer to the Transaction Specification.

The work splits along these lines:

builder            Transaction: an operation plus the version it was based on
operation          the vocabulary of changes an operation can describe
update_map         incremental edits to the manifest's string maps
validate           pre-commit checks against the manifest being replaced
prepare            what a commit attempt settles about the index list first
manifest_build     applying an operation to produce the next manifest
index_maintenance  how that narrows or drops index metadata
row_version        how it assigns row ids and per-row version metadata
conflicts          whether two operations collide, for the commit retry path
proto              the persisted protobuf encoding of all of the above

Structs

CoverageIdentity

What one physical index segment contributes to coverage.

DataOverlayGroup

Overlay files to append to a single fragment, in order (the last entry is newest). The overlays are appended to the fragment’s existing overlays list rather than replacing it, so overlays written by concurrent commits are preserved. Each overlay’s committed_version is stamped to the new dataset version at commit time (re-stamped on retry).

DataReplacementGroup

PreparedIndices

The index lists of one commit attempt: the list as read from the manifest current at the attempt, the prepared list the builder starts from, and what was settled about the fragment reuse entry.

ReadVersionState

The version a transaction read, as the coverage derivation needs it.

RewriteGroup

RewrittenIndex

TaggedRewriteAssembly

What the commit path assembled for a rewrite on a tagged fragment reuse history: the complete entry to install, merged against the manifest the commit attempt builds on. Supplied through FragReuseUpdate::Rewrite on the prepared index list once per attempt, while the operation’s frag_reuse_index stays what the caller supplied, so every retry works from the same intent.

Transaction

A change to a dataset that can be retried

TransactionBuilder

Add TransactionBuilder for flexibly setting option without using mut

UpdateMap

Represents updates to a map (either incremental or replacement)

UpdateMapEntry

An entry for a map update. If value is None, the key will be removed from the map.

UpdatedFragmentOffsets

Matched physical row offsets per fragment for a partial UpdateMode::RewriteColumns update.

Enums

FragReuseUpdate

What the commit path settled about the fragment reuse entry for one attempt. In-memory only, like the rewrite intent it carries.

Operation

An operation on a dataset.

UpdateMode

Constants

RANGE_SEGMENTS_CONFIG_KEY

Table config key that opts a table into writing clustered deletions as Range segments (U64Segment::Ranges). Set it to true with update_config.

Functions

has_writer_placed_lineage

Whether an update’s new fragment carries row lineage its writer placed outside the manifest, which the commit keeps instead of resolving.

reordered_sources

The union of transitions’ source fragment ids. Rewrite groups covered by this set skip index-bitmap maintenance: their bitmaps keep the retired source ids as provenance.

translate_config_updates

Helper function to translate old-style config updates to new UpdateMap format

translate_schema_metadata_updates

Helper function to translate old-style schema metadata to new UpdateMap format

validate_operation

Validate the operation is valid for the given manifest.

Type Aliases

LogicalIndexSegments

Non-system logical index name -> its physical segments, ordered by UUID.