Skip to main content Module segment Copy item path Source AsyncStoreReader Async document store reader - loads blocks on demand EagerParallelStoreWriter Parallel document store writer - compresses blocks immediately when queued FieldStats Field statistics for BM25F scoring FlatVectorData Flat vector binary format helpers for writing. IVFRaBitQIndexData IVF-RaBitQ index data (codebook + cluster assignments) LazyFlatVectorData Lazy flat vector data — zero-copy doc_id index, vectors via range reads. MemoryBreakdown Detailed memory breakdown by component MergeStats Statistics for merge operations RawStoreBlock Raw block info for store merging (without decompression) ScaNNIndexData ScaNN index data (codebook + cluster assignments) SegmentBuilder Segment builder with optimized memory usage SegmentBuilderConfig Configuration for segment builder SegmentBuilderStats Statistics about segment builder state SegmentFiles Paths for segment files SegmentId Unique segment identifier (UUID7-like: 48-bit timestamp + 80-bit random) SegmentMerger Segment merger - merges multiple segments into one SegmentMeta Segment metadata SegmentReader Async segment reader with lazy loading SegmentSnapshot RAII guard that holds references to a snapshot of segments.
When dropped, releases all segment references and triggers deferred deletion. SegmentTracker Tracks segment references and pending deletions SparseIndex Sparse vector index for a field: lazy block loading via mmap. StoreMerger Store merger - concatenates compressed blocks from multiple stores without recompression TrainedVectorStructures Trained vector index structures for rebuilding segments with ANN indexes VectorSearchResult Vector search result with ordinal tracking for multi-value fields VectorIndex Vector index type - RaBitQ, IVF-RaBitQ, or ScaNN (IVF-PQ) DEFAULT_DICT_SIZE Default dictionary size (4KB is a good balance) STORE_BLOCK_SIZE Block size for document store (16KB).
Smaller blocks reduce read amplification for single-doc fetches at the
cost of slightly worse compression ratio. Zstd dictionary training
recovers most of the compression loss. delete_segment Delete segment files from directory (all deletions run concurrently). dequantize_raw Dequantize raw bytes to f32 based on storage quantization. deserialize_document Deserialize all fields from document bytes. deserialize_document_fields Deserialize only specific fields from document bytes. serialize_document Binary document format:
num_fields: u16
per field: field_id: u16, type_tag: u8, value data
0=Text: len:u32 + utf8
1=U64: u64 LE
2=I64: i64 LE
3=F64: f64 LE
4=Bytes: len:u32 + raw
5=SparseVector: count:u32 + count*(u32+f32)
6=DenseVector: count:u32 + count*f32
7=Json: len:u32 + json utf8 serialize_document_into Serialize a document into a reusable buffer (clears it first).
Avoids per-document allocation when called in a loop.