Struct SegmentMerger 

pub struct SegmentMerger { /* private fields */ }

Segment merger - merges multiple segments into one

Implementations

impl SegmentMerger

pub async fn compact<D: DirectoryWriter>( &self, dir: &D, source: &SegmentReader, output: SegmentId, memory_budget: usize, ) -> Result<(SegmentMeta, MergeStats)>

Remove deleted rows from one segment. The caller owns input/output IDs and publishes the result. memory_budget bounds decoded scratch, in addition to the immutable source reader’s documented residency.

impl SegmentMerger

pub fn new(schema: Arc<Schema>) -> Self

pub fn with_term_dict_block_size(self, size: SSTableBlockSize) -> Self

Set the validated flush target for newly written term dictionaries.

pub fn with_posting_config( self, optimization: IndexOptimization, posting_codec: PostingCodec, ) -> Self

Configure posting compression for newly encoded output.

pub fn with_reorder_fields(self, reorder: bool) -> Self

Enable field-local BP during merge (historically BMP-only). Text and BMP fields still opt in through their schema reorder flag.

pub fn with_background_pool(self, pool: Option<Arc<ThreadPool>>) -> Self

Run merge-time BP on this bounded pool instead of the global one.

pub fn with_granularity(self, granularity: BpGranularity) -> Self

Set merge-time BP granularity (see granularity).

pub fn with_bp_budget(self, budget: BpBudget) -> Self

Bound merge-time BP wall clock (see bp_budget).

pub fn with_bp_memory_budget(self, bytes: usize) -> Self

Memory budget for the BP forward index (see bp_memory_budget).

pub fn with_reorder_permits(self, permits: Arc<ReorderConcurrencyGate>) -> Self

Share the application-wide whole-segment reorder gate.

pub async fn merge<D: Directory + DirectoryWriter>( &self, dir: &D, segments: &[SegmentReader], new_segment_id: SegmentId, trained: Option<&TrainedVectorStructures>, ) -> Result<(SegmentMeta, MergeStats)>

Merge segments into one, streaming postings/positions/store directly to files.

If trained is provided, dense vectors use O(1) cluster merge when possible (compatible IVF-PQ), otherwise rebuilds ANN from global artifacts. Without trained structures, only flat vectors are merged.

Uses streaming writers so postings, positions, and store data flow directly to files instead of buffering everything in memory. Only the term dictionary (compact key+TermInfo entries) is buffered.

This is the physical encoded-copy primitive. Readers with deletion masks must use compact, or the index writer’s merge operation, which owns visibility capture and atomic publication across multiple sources.