Struct SegmentBuilder 

pub struct SegmentBuilder { /* private fields */ }

Segment builder with optimized memory usage

Features:

• Streams documents to disk immediately (no in-memory document storage)
• Uses string interning for terms (reduced allocations)
• Uses hashbrown HashMap (faster than BTreeMap)

Implementations

impl SegmentBuilder

pub fn new(schema: Schema, config: SegmentBuilderConfig) -> Result<Self>

Create a new segment builder

pub fn set_tokenizer(&mut self, field: Field, tokenizer: BoxedTokenizer)

pub fn num_docs(&self) -> u32

pub fn estimated_memory_bytes(&self) -> usize

Fast O(1) memory estimate - updated incrementally during indexing

pub fn recalibrate_memory(&mut self)

Recalibrate incremental memory estimate using capacity-based calculation. More expensive than estimated_memory_bytes() — O(terms + dims) vs O(1) — but accounts for Vec capacity growth (doubling) and HashMap table overhead. Call periodically (e.g. every 1000 docs) to prevent drift.

pub fn sparse_dim_count(&self) -> usize

Count total unique sparse dimensions across all fields

pub fn stats(&self) -> SegmentBuilderStats

Get current statistics for debugging performance (expensive - iterates all data)

pub fn add_document(&mut self, doc: Document) -> Result<DocId>

Add a document - streams to disk immediately

pub async fn build<D: Directory + DirectoryWriter>( self, dir: &D, segment_id: SegmentId, ) -> Result<SegmentMeta>

Build the final segment

Streams all data directly to disk via StreamingWriter to avoid buffering entire serialized outputs in memory. Each phase consumes and drops its source data before the next phase begins.

Trait Implementations

impl Drop for SegmentBuilder

fn drop(&mut self)

Executes the destructor for this type.