Skip to main content

Module row_addr_remap

Module row_addr_remap 

Source
Expand description

Compact row-address remapping for compaction.

Compaction rewrites rows into new fragments, so indices that store physical row addresses need an old-address to new-address mapping without building an O(total rows) HashMap<u64, Option<u64>>.

Layout:

  • Old rows: old_fragment_id -> (old_offsets, old_rows_before)
    • old_offsets: rewritten old row offsets in this old fragment.
    • old_rows_before: rewritten row count before this old fragment.
  • New rows: ordered new-fragment ranges (fragment_id, new_rows_before, physical_rows)
    • new_rows_before: rewritten row count before this new fragment.

Lookup:

  • An address whose fragment was not rewritten returns None.
  • For an address whose fragment was rewritten:
    • Read (old_offsets, old_rows_before) from the old-row layout.
    • If offset is not in old_offsets, return Some(None) because the row was deleted.
    • Otherwise, old_offsets.rank(offset) - 1 is this row’s 0-based position among rewritten old rows in this old fragment. Add old_rows_before to get k, the row’s 0-based position among all rewritten old rows.
    • In the new-row layout, find the range (fragment_id, new_rows_before, physical_rows) where new_rows_before <= k < new_rows_before + physical_rows.
    • The new address is (fragment_id, k - new_rows_before).

Ordering:

Compact remap does not store each old-to-new row mapping. It computes k from the old-row layout, then maps it to the k-th row written to the new fragments. This requires the reader-to-writer pipeline to preserve row order.

  • old_frag_ids must match the order old fragments are read. Within each old fragment, rewritten rows are interpreted by ascending old row offset.
  • new_frags must match the order new rows are written.
  • Current compaction satisfies this because it scans selected fragments in order and writes the resulting stream without reordering rows.

Structs§

CompactRowAddrRemap
Compact remap backed by per-group rewritten row bitmaps + new-fragment layouts.
GroupInput
Input describing one rewrite group: the old row addresses that were rewritten plus the fragment layout before/after the rewrite.

Enums§

RowAddrRemap
A queryable row-address remapping with the exact semantics of HashMap<u64, Option<u64>>::get(&addr).copied():