Module batch_segment_writer 

True Batch SegmentWriter Ingest

High-performance batch API for segment construction that eliminates per-vector overhead and enables streaming ingest.

Problem

Current SegmentWriter.add():

• Creates vec_owned per vector (allocation)
• Rotates synchronously (CPU bound)
• Single vector at a time (no batching benefits)

Solution

Batch-oriented API:

• add_batch_contiguous() for bulk ingest
• Pre-allocate rotation buffers
• Parallel rotation for batch
• Direct arena storage

Usage

use sochdb_vector::batch_segment_writer::{BatchSegmentWriter, BatchConfig};

let config = BatchConfig::default();
let mut writer = BatchSegmentWriter::new(config);

// Batch ingest
let vectors: &[f32] = &flat_data;
let ids = writer.add_batch_contiguous(vectors, dim, keys)?;

// Build segment
let segment = writer.build()?;

Structs

BatchConfig

Configuration for batch segment writer

BatchSegmentWriter

High-performance batch segment writer

BatchWriteStats

Batch write statistics

BuiltSegment

Built segment ready for use

StoredVector

Stored vector with metadata

Enums

BatchWriteError

Error type for batch operations

Type Aliases

VectorKey

Key type for vectors