Struct SegmentBuilder 

pub struct SegmentBuilder { /* private fields */ }

Segment builder with optimized memory usage

Features:

• Streams documents to disk immediately (no in-memory document storage)
• Uses string interning for terms (reduced allocations)
• Uses hashbrown HashMap (faster than BTreeMap)

Implementations

impl SegmentBuilder

pub fn new(schema: Schema, config: SegmentBuilderConfig) -> Result<Self>

Create a new segment builder

pub fn set_tokenizer(&mut self, field: Field, tokenizer: BoxedTokenizer)

pub fn num_docs(&self) -> u32

pub fn estimated_memory_bytes(&self) -> usize

Fast O(1) memory estimate - updated incrementally during indexing

pub fn recalibrate_memory(&mut self)

Recalibrate incremental memory estimate using capacity-based calculation. More expensive than estimated_memory_bytes() — O(terms + dims) vs O(1) — but accounts for Vec capacity growth (doubling) and HashMap table overhead. Call periodically (e.g. every 1000 docs) to prevent drift.

pub fn sparse_dim_count(&self) -> usize

Count total unique sparse dimensions across all fields

pub fn stats(&self) -> SegmentBuilderStats

Get current statistics for debugging performance (expensive - iterates all data)

pub fn add_document(&mut self, doc: Document) -> Result<DocId>

Add a document - streams to disk immediately

pub async fn build<D: Directory + DirectoryWriter>( self, dir: &D, segment_id: SegmentId, ) -> Result<SegmentMeta>

Build the final segment

Memory optimization: each phase consumes and drops its source data before the next phase begins, preventing accumulation of multiple large buffers.

Trait Implementations

impl Drop for SegmentBuilder

fn drop(&mut self)

Executes the destructor for this type.