Struct SegmentManager 

pub struct SegmentManager { /* private fields */ }

Segment manager for a collection

Implementations

impl SegmentManager

pub fn new(collection: impl Into<String>) -> SegmentManager

Create a new segment manager

pub fn with_config( collection: impl Into<String>, config: ManagerConfig, ) -> SegmentManager

Create with custom configuration

pub fn get_or_init_schema( &self, named: &HashMap<String, Value>, ) -> Arc<Vec<String>>

Get or create the shared column schema from first row’s named fields.

pub fn column_schema(&self) -> Option<Arc<Vec<String>>>

Get the column schema if it exists.

pub fn collection(&self) -> &str

Get collection name

pub fn config(&self) -> &ManagerConfig

Get configuration

pub fn stats(&self) -> ManagerStats

Get statistics. total_entities is read from the lock-free atomic; other fields come from the slow-path stats struct.

pub fn next_entity_id(&self) -> EntityId

Generate a new entity ID

pub fn next_row_id(&self) -> u64

Generate a per-table sequential row ID (1, 2, 3… per collection)

pub fn reserve_row_ids(&self, n: u64) -> Range<u64>

Reserve n contiguous per-table row IDs with one atomic fetch_add. Caller assigns row_id = start + i per entity. Saves N-1 atomic RMWs on bulk inserts (25k atomics → 1).

pub fn register_row_id(&self, id: u64)

Advance the per-table row_id counter to at least id + 1. Called during load to restore the counter from existing data.

pub fn insert(&self, entity: UnifiedEntity) -> Result<EntityId, SegmentError>

Insert a new entity

pub fn insert_batch( &self, entities: Vec<UnifiedEntity>, ) -> Result<Vec<EntityId>, SegmentError>

Insert multiple entities (batch) — sequential, one lock per item.

pub fn bulk_insert( &self, entities: Vec<UnifiedEntity>, ) -> Result<Vec<EntityId>, SegmentError>

Turbo bulk insert — single lock acquisition for the entire batch. Skips bloom filter, memtable, and cross-ref indexing for maximum speed.

pub fn get(&self, id: EntityId) -> Option<UnifiedEntity>

Get an entity by ID — scans growing then sealed segments.

pub fn get_many(&self, ids: &[EntityId]) -> Vec<Option<UnifiedEntity>>

Batch-fetch multiple entities by ID in a single lock acquisition per segment.

For indexed-scan result sets (up to ~5000 ids from range/bitmap lookup) this is 2-3 lock acquisitions total vs N×3 with individual get() calls.

pub fn for_each_id<F>(&self, ids: &[EntityId], f: F)

where F: FnMut(usize, &UnifiedEntity),

Visitor-pattern batch fetch. Invokes f(&UnifiedEntity, usize_index) for each id that resolves, never cloning the entity.

Used by scan hot paths (select_range, select_filtered) that materialize each entity into an output record and don’t need an owned UnifiedEntity. Eliminates ~20% of scan CPU spent in UnifiedEntity::clone when get_batch is followed by runtime_table_record_lean(entity).

The closure runs while the segment read lock is held, so it must be short — avoid doing I/O or taking unrelated locks in f.

pub fn update(&self, entity: UnifiedEntity) -> Result<(), SegmentError>

Update an entity

pub fn update_with_metadata( &self, entity: UnifiedEntity, metadata: Option<&Metadata>, ) -> Result<(), SegmentError>

Update an entity and, optionally, replace its metadata while holding the segment write lock only once.

pub fn update_hot( &self, entity: UnifiedEntity, modified_columns: &[String], ) -> Result<(), SegmentError>

HOT-update: like update but skips index work for unchanged columns. modified_columns is the list of column names actually changed by the UPDATE statement — lets us skip pk_index and cross_ref when safe.

pub fn update_hot_with_metadata( &self, entity: UnifiedEntity, modified_columns: &[String], metadata: Option<&Metadata>, ) -> Result<(), SegmentError>

HOT-update an entity and, optionally, replace its metadata while holding the segment write lock only once.

pub fn update_hot_batch_with_metadata<'a, I>( &self, items: I, ) -> Result<(), SegmentError>

where I: IntoIterator<Item = (&'a UnifiedEntity, &'a [String], Option<&'a Metadata>)>,

Batch HOT-update multiple entities while holding the growing-segment write lock only once when possible.

pub fn delete(&self, id: EntityId) -> Result<bool, SegmentError>

Delete an entity

pub fn delete_batch( &self, ids: &[EntityId], ) -> Result<Vec<EntityId>, SegmentError>

pub fn get_metadata(&self, id: EntityId) -> Option<Metadata>

Get metadata for an entity

pub fn set_metadata( &self, id: EntityId, metadata: Metadata, ) -> Result<(), SegmentError>

Set metadata for an entity

pub fn seal_current(&self) -> Result<u64, SegmentError>

Seal the current growing segment

pub fn force_seal(&self) -> Result<Option<u64>, SegmentError>

Force seal (for testing/manual control)

pub fn all_visible_fraction(&self) -> f64

Fraction of “pages” in sealed segments that are marked all-visible.

Sealed segments are immutable so all their rows are safe for index-only scans. The growing segment is never counted (writes may be in-flight). Uses rows_per_page = 256 (matching 8 KB pages with ~32-byte rows).

Returns a value in [0.0, 1.0]. 1.0 when all sealed rows are visible; 0.0 when there are no sealed segments.

pub fn for_each_entity<F>(&self, callback: F)

where F: FnMut(&UnifiedEntity) -> bool,

Iterate over all entities in-place without collecting into a Vec.

The callback receives a reference to each entity. Return true to continue iteration, false to stop early (e.g. when a LIMIT is reached). This avoids the allocation and cloning overhead of query_all.

pub fn fold_entities_parallel<T, FInit, FFold, FReduce>( &self, init: FInit, fold: FFold, reduce: FReduce, ) -> T

where T: Send, FInit: Fn() -> T + Send + Sync, FFold: Fn(T, &UnifiedEntity) -> T + Send + Sync, FReduce: Fn(T, T) -> T + Send + Sync,

Parallel fold across all entities. Each sealed segment is processed on its own rayon task; the growing segment stays on the caller thread (its read lock is briefly held).

• init builds a fresh accumulator per thread.
• fold mutates an accumulator with one entity at a time.
• reduce combines two accumulators into one.

The returned value is the reduction of every per-thread accumulator. Use this for aggregate-shape workloads (GROUP BY) where per-thread partial state can be merged cheaply.

NOTE: when there are 0 or 1 sealed segments, the parallel path is skipped and the work runs sequentially to avoid rayon overhead on tiny tables.

pub fn for_each_entity_zoned<F>( &self, zone_preds: &[(&str, ZoneColPred<'_>)], callback: F, )

where F: FnMut(&UnifiedEntity) -> bool,

Zone-map-aware iteration across all segments.

Like for_each_entity, but checks zone_preds against each segment’s column zone maps before iterating. Segments where any predicate can definitively prove no rows match are skipped entirely.

zone_preds: slice of (column_name, ZoneColPred) extracted from the WHERE clause. Empty slice → same behaviour as for_each_entity (no pruning).

pub fn query_all_zoned<F>( &self, zone_preds: &[(&str, ZoneColPred<'_>)], filter: F, ) -> Vec<UnifiedEntity>

where F: Fn(&UnifiedEntity) -> bool + Sync,

Zone-map-aware parallel query.

Like query_all but applies zone_preds on the main thread to prune sealed segments before spawning workers — segments that provably contain no matching rows are skipped entirely.

Zone check runs single-threaded (it reads per-segment metadata, not row data), so it’s cheap. Surviving segments are then scanned in parallel using std::thread::scope when there are > 1 of them.

pub fn query_all<F>(&self, filter: F) -> Vec<UnifiedEntity>

where F: Fn(&UnifiedEntity) -> bool + Sync,

Query across all segments. Uses parallel scanning for sealed segments when more than one sealed segment exists.

pub fn query_with_bloom_hint<F>( &self, key_hint: Option<&[u8]>, filter: F, ) -> (Vec<UnifiedEntity>, bool)

where F: Fn(&UnifiedEntity) -> bool,

Query with bloom filter hint: skip the growing segment when bloom says key is absent.

This is the integration point for bloom filter pruning. When a query has an equality predicate on a known key, the executor can call this instead of query_all to avoid scanning when the bloom filter proves the key doesn’t exist.

Returns (results, bloom_pruned) where bloom_pruned indicates if the segment was skipped.

pub fn filter_metadata( &self, filters: &[(String, MetadataFilter)], ) -> Vec<EntityId>

Filter by metadata across all segments

pub fn get_by_kind(&self, kind: &str) -> Vec<UnifiedEntity>

Get entities by kind

pub fn count(&self) -> usize

Count entities

pub fn segment_ids(&self) -> Vec<u64>

Get all segment IDs

pub fn drain_events(&self) -> Vec<LifecycleEvent>

Drain events. Kept for API compatibility; always returns empty because emit no longer buffers.

pub fn run_maintenance(&self) -> Result<(), SegmentError>

Run maintenance (would be called periodically in production)