Struct SegmentBuilder 

pub struct SegmentBuilder { /* private fields */ }

Segment builder with optimized memory usage

Features:

• Streams documents to disk immediately (no in-memory document storage)
• Uses string interning for terms (reduced allocations)
• Uses hashbrown HashMap (faster than BTreeMap)

Implementations

impl SegmentBuilder

pub fn new(schema: Arc<Schema>, config: SegmentBuilderConfig) -> Result<Self>

Create a new segment builder

pub fn set_tokenizer(&mut self, field: Field, tokenizer: BoxedTokenizer)

pub fn num_docs(&self) -> u32

pub fn estimated_memory_bytes(&self) -> usize

Fast O(1) memory estimate - updated incrementally during indexing

pub fn sparse_dim_count(&self) -> usize

Count total unique sparse dimensions across all fields

pub fn stats(&self) -> SegmentBuilderStats

Get current statistics for debugging performance (expensive - iterates all data)

pub fn add_document(&mut self, doc: Document) -> Result<DocId>

Add a document - streams to disk immediately

pub async fn build<D: Directory + DirectoryWriter>( self, dir: &D, segment_id: SegmentId, trained: Option<&TrainedVectorStructures>, ) -> Result<SegmentMeta>

Build the final segment

Streams all data directly to disk via StreamingWriter to avoid buffering entire serialized outputs in memory. Each phase consumes and drops its source data before the next phase begins.

Trait Implementations

impl Drop for SegmentBuilder

Available on crate feature native only.

fn drop(&mut self)

Executes the destructor for this type.

fn pin_drop(self: Pin<&mut Self>)

🔬This is a nightly-only experimental API. (pin_ergonomics)

Execute the destructor for this type, but different to Drop::drop, it requires self to be pinned.