Skip to main content Module segment Copy item path Source AsyncSegmentReader Async segment reader with lazy loading AsyncStoreReader Async document store reader - loads blocks on demand EagerParallelStoreWriter Parallel document store writer - compresses blocks immediately when queued FieldStats Field statistics for BM25F scoring FlatVectorData Flat vector binary format helpers for writing. IVFRaBitQIndexData IVF-RaBitQ index data with embedded centroids and codebook LazyFlatVectorData Lazy flat vector data — doc_ids in memory, vectors accessed via range reads. MemoryBreakdown Detailed memory breakdown by component MergeStats Statistics for merge operations RawStoreBlock Raw block info for store merging (without decompression) ScaNNIndexData ScaNN index data with embedded centroids and codebook SegmentBuilder Segment builder with optimized memory usage SegmentBuilderConfig Configuration for segment builder SegmentBuilderStats Statistics about segment builder state SegmentFiles Paths for segment files SegmentId Unique segment identifier (UUID7-like: 48-bit timestamp + 80-bit random) SegmentMerger Segment merger - merges multiple segments into one SegmentMeta Segment metadata SegmentSnapshot RAII guard that holds references to a snapshot of segments
When dropped, releases all segment references and deletes any
segments that were pending deletion and have no remaining references. SegmentTracker Tracks segment references and pending deletions SparseIndex Sparse vector index for a field: lazy block loading via mmap StoreMerger Store merger - concatenates compressed blocks from multiple stores without recompression TrainedVectorStructures Trained vector index structures for rebuilding segments with ANN indexes VectorSearchResult Vector search result with ordinal tracking for multi-value fields VectorIndex Vector index type - RaBitQ, IVF-RaBitQ, or ScaNN (IVF-PQ) DEFAULT_DICT_SIZE Default dictionary size (64KB is a good balance) STORE_BLOCK_SIZE Block size for document store (256KB for better compression)
Larger blocks = better compression ratio but more memory per block load delete_segment Delete segment files from directory deserialize_document serialize_document Binary document format:
num_fields: u16
per field: field_id: u16, type_tag: u8, value data
0=Text: len:u32 + utf8
1=U64: u64 LE
2=I64: i64 LE
3=F64: f64 LE
4=Bytes: len:u32 + raw
5=SparseVector: count:u32 + count*(u32+f32)
6=DenseVector: count:u32 + count*f32
7=Json: len:u32 + json utf8 SegmentReader Alias for AsyncSegmentReader