Struct MutationEngine 

pub struct MutationEngine { /* private fields */ }

Coordinates all columnar mutations for a single collection.

Owns the memtable, PK index, and per-segment delete bitmaps. Produces WAL records for each mutation that the caller must persist.

Implementations

impl MutationEngine

pub fn new(collection: String, schema: ColumnarSchema) -> Self

Create a new mutation engine for a collection.

pub fn memtable(&self) -> &ColumnarMemtable

Access the memtable.

pub fn memtable_mut(&mut self) -> &mut ColumnarMemtable

Mutable access to the memtable (for drain on flush).

pub fn pk_index(&self) -> &PkIndex

Access the PK index.

pub fn pk_index_mut(&mut self) -> &mut PkIndex

Mutable access to the PK index (for cold-start rebuild).

pub fn delete_bitmap(&self, segment_id: u64) -> Option<&DeleteBitmap>

Access a segment’s delete bitmap.

pub fn delete_bitmap_mut(&mut self, segment_id: u64) -> &mut DeleteBitmap

Mutable access to a segment’s delete bitmap. Creates an empty one on first access so callers can mark_deleted_batch unconditionally. Used by temporal-purge paths that tombstone superseded row positions without going through the single-row insert / delete paths.

pub fn memtable_segment_id(&self) -> u64

The virtual segment id used for rows still in the memtable.

pub fn pk_col_indices(&self) -> &[usize]

The schema’s primary-key column indices, in schema order.

pub fn delete_bitmaps(&self) -> &HashMap<u64, DeleteBitmap>

Access all delete bitmaps.

pub fn collection(&self) -> &str

The collection name.

pub fn schema(&self) -> &ColumnarSchema

The schema.

pub fn should_flush(&self) -> bool

Whether the memtable should be flushed.

pub fn memtable_surrogates(&self) -> &[Option<Surrogate>]

Access the per-row surrogate table for the memtable.

Index matches memtable row order; None entries indicate rows inserted without a surrogate (test fixtures, legacy paths).

pub fn scan_memtable_rows(&self) -> impl Iterator<Item = Vec<Value>> + '_

Iterate non-deleted rows in the memtable as Vec<Value>.

Skips rows marked as deleted in the memtable’s virtual segment delete bitmap. For rows in flushed segments, use SegmentReader.

pub fn scan_memtable_rows_with_surrogates( &self, ) -> impl Iterator<Item = (Option<Surrogate>, Vec<Value>)> + '_

Iterate non-deleted rows paired with their surrogate identity.

Yields (Option<Surrogate>, Vec<Value>). The surrogate is None for rows inserted without one (test fixtures, legacy paths). Deleted rows are filtered out exactly as in Self::scan_memtable_rows.

pub fn get_memtable_row(&self, row_idx: usize) -> Option<Vec<Value>>

Get a single row from the memtable by index (None if deleted).

pub fn rollback_memtable_inserts( &mut self, row_count_before: usize, inserted_pks: &[Vec<u8>], displaced: &[(Vec<u8>, RowLocation)], )

Roll back in-memory inserts to row_count_before.

Undoes the effect of one or more inserts that appended rows starting at row_count_before. For each inserted row:

• The corresponding PK entry is removed from the PK index.
• If the insert displaced a prior row (upsert tombstone), that prior row’s PK index entry is restored and its tombstone bit cleared.

The memtable is then truncated to row_count_before. Used exclusively by the transaction undo log; never called on the normal write path.

pub fn next_segment_id(&self) -> u64

The segment ID that will be assigned to the next flushed segment.

Use this to obtain the ID to pass to on_memtable_flushed.

pub fn should_compact(&self, segment_id: u64, total_rows: u64) -> bool

Whether a segment should be compacted based on its delete ratio.

pub fn encode_pk_from_row( &self, values: &[Value], ) -> Result<Vec<u8>, ColumnarError>

Encode a PK value as index bytes. Exposed for callers that need to probe the PK index (e.g. ON CONFLICT DO UPDATE routing).

impl MutationEngine

pub fn on_memtable_flushed( &mut self, new_segment_id: u64, ) -> Result<MutationResult, ColumnarError>

Notify the engine that the memtable was flushed to a new segment.

Updates the PK index to remap memtable entries to the new segment. Returns the WAL record for the flush event, or SegmentIdExhausted if the u64 segment ID counter has wrapped past its maximum.

pub fn on_compaction_complete( &mut self, old_segment_ids: &[u64], new_segment_id: u64, row_mapping: &HashMap<(u64, u32), u32>, ) -> MutationResult

Notify the engine that compaction completed.

Remaps PK index entries and removes old delete bitmaps.

impl MutationEngine

pub fn insert( &mut self, values: &[Value], ) -> Result<MutationResult, ColumnarError>

Insert a row with upsert-on-duplicate semantics. Returns WAL records to persist.

Validates schema. If the PK already exists, the prior row is tombstoned via the segment’s delete bitmap (a single positional delete) before the new row is appended to the memtable. The PK index is rebound to the new row location. This matches the ClickHouse / Iceberg “sparse PK + positional delete” model and keeps SELECT WHERE pk = X linearizable on one row without a read-time merge pass.

Callers that want strict INSERT (error on duplicate) should check pk_index().contains() themselves before calling; callers that want ON CONFLICT DO NOTHING semantics should use Self::insert_if_absent.

pub fn insert_with_surrogate( &mut self, values: &[Value], surrogate: Surrogate, ) -> Result<MutationResult, ColumnarError>

Insert with a stable cross-engine surrogate identity.

Identical to Self::insert but also records surrogate in the per-row side-table so scan prefilters can perform bitmap membership checks without a separate lookup pass.

pub fn insert_if_absent( &mut self, values: &[Value], ) -> Result<MutationResult, ColumnarError>

INSERT ... ON CONFLICT DO NOTHING semantics: append only if the PK is absent; silently skip on duplicate.

Returns Ok(MutationResult { wal_records }) with an empty vector when the row was skipped, so callers that batch WAL appends can detect no-ops by checking wal_records.is_empty().

pub fn lookup_memtable_row_by_pk(&self, pk_bytes: &[u8]) -> Option<Vec<Value>>

Look up the current row for a PK in the memtable, if present.

Returns None if the PK is not in the index, or if the PK points to a flushed segment (callers needing cross-segment lookup must go through a segment reader separately). This is the fast path used by ON CONFLICT DO UPDATE to read the would-be-merged row when the duplicate hits the memtable — the common case under back-to-back inserts.

pub fn delete( &mut self, pk_value: &Value, ) -> Result<MutationResult, ColumnarError>

Delete a row by PK value. Returns WAL record to persist.

Looks up PK in the index to find the segment + row, then marks the row in the segment’s delete bitmap.

pub fn update( &mut self, old_pk: &Value, new_values: &[Value], ) -> Result<MutationResult, ColumnarError>

Update a row by PK: DELETE old + INSERT new.

updates maps column names to new values. Columns not in the map retain their existing values from the old row.

Returns WAL records for both the delete and the insert.

NOTE: The caller must provide the full old row values for the re-insert. This method takes the complete new row (already merged with old values).