Struct CompactionExecutor 

pub struct CompactionExecutor { /* private fields */ }

Compaction executor that performs merge-sort compaction.

Reads all input segments, merge-sorts by (series_key, timestamp), deduplicates, optionally removes tombstoned data, and writes an optimized output segment.

Tombstone lookup complexity

Tombstones are passed as &TombstoneSet, which uses a HashMap keyed by canonical series name for O(1) amortised per-row lookup in execute. Each entry supports both full-series and ranged (time-window) deletes. The overall cleanup pass is therefore O(rows), not O(rows × tombstones).

Implementations

impl CompactionExecutor

pub fn new( compress: bool, float_encoding: FloatEncoding, row_group_size: usize, compression_codec: CompressionCodec, zstd_level: i32, ) -> Self

Create an executor with custom settings.

pub fn with_field_encryption( self, writer: FieldEncryptionConfig, reader: Arc<dyn FieldKeyProvider>, ) -> Self

Give the executor this database’s per-column encryption.

Both halves, or neither. A compaction reads input segments and writes a new one, so a key provider without the writer config decrypts on the way in and writes the column back in plaintext — a compaction pass that silently undoes the encryption, hours after the write that asked for it, with no error anywhere. The writer config alone is the mirror image: the read fails closed, which is merely broken rather than dangerous.

pub fn with_task_timeout(self, timeout: Duration) -> Self

Set the maximum wall-clock time for a single compaction task.

pub fn with_max_input_bytes(self, limit: u64) -> Self

Set the maximum total bytes for input segments.

When the cumulative in-memory size of loaded RecordBatches exceeds this threshold the compaction task is rejected early with CompactionError::InputTooLarge. Pass 0 for unlimited.

pub fn execute( &self, task: &CompactionTask, tombstones: &TombstoneSet, ) -> Result<SegmentMeta>

Execute a compaction task.

Reads all input segments, merges their data, deduplicates on (series_key_hash, timestamp, canonical), removes tombstoned series, and writes an optimized output segment.

K-way Merge Architecture

Each segment’s row indices are sorted independently by (series_hash, timestamp, canonical_id), then merged using a K-element BinaryHeap (min-heap) in O(N log K) time — versus the previous O(N log N) global sort. Deduplication is inline: duplicate (hash, ts, canonical) keys are resolved by keeping only the row from the newest segment (last-write-wins).

Output is assembled column-by-column via arrow::compute::interleave in row-group-sized chunks, bounding peak memory to O(chunk_size × columns) for the output stage.

Errors

Returns an error if no eligible segments exist, reading/writing segments fails, or deduplication encounters an internal error.

Trait Implementations

impl Default for CompactionExecutor

fn default() -> Self

Returns the “default value” for a type.