Struct NodeStore 

pub struct NodeStore { /* private fields */ }

Persistent node property store rooted at a database directory.

On-disk layout:

{root}/nodes/{label_id}/hwm.bin            — high-water mark (u64 LE)
{root}/nodes/{label_id}/col_{col_id}.bin   — flat u64 column array
{root}/strings.bin                         — overflow string heap (SPA-212)

The overflow heap is an append-only byte file. Each entry is a raw byte sequence (no length prefix); the offset and length are encoded into the TAG_BYTES_OVERFLOW u64 stored in the column file.

Implementations

impl NodeStore

pub fn open(db_root: &Path) -> Result<Self>

Open (or create) a node store rooted at db_root.

pub fn root_path(&self) -> &Path

Return the root directory path of this node store.

pub fn encode_value(&self, val: &Value) -> Result<u64>

Encode a Value for column storage, writing long Bytes strings to the overflow heap (SPA-212).

• Int64 → identical to Value::to_u64().
• Bytes ≤ 7 B → inline TAG_BYTES encoding, identical to Value::to_u64().
• Bytes > 7 B → appended to strings.bin; returns TAG_BYTES_OVERFLOW u64.
• Float → 8 raw IEEE-754 bytes appended to strings.bin; returns a TAG_FLOAT u64 so all 64 float bits are preserved (SPA-267).

pub fn decode_raw_value(&self, raw: u64) -> Value

Decode a raw u64 column value back to a Value, reading the overflow string heap when the tag is TAG_BYTES_OVERFLOW or TAG_FLOAT (SPA-212, SPA-267).

Handles all four tags:

• TAG_INT64 → Value::Int64
• TAG_BYTES → Value::Bytes (inline, ≤ 7 bytes)
• TAG_BYTES_OVERFLOW → Value::Bytes (from heap)
• TAG_FLOAT → Value::Float (8 raw IEEE-754 bytes from heap)

pub fn raw_str_matches(&self, raw: u64, s: &str) -> bool

Check whether a raw stored u64 encodes a string equal to s.

Handles both inline (TAG_BYTES) and overflow (TAG_BYTES_OVERFLOW) encodings (SPA-212). Used by WHERE-clause and prop-filter comparison.

pub fn flush_hwms(&mut self) -> Result<()>

Persist all dirty in-memory HWMs to disk atomically.

Called once per transaction commit rather than once per node creation, so that bulk imports do not incur one fsync per node (SPA-217 regression fix).

Each dirty label’s HWM is written via the same tmp+fsync+rename strategy used by [save_hwm], preserving the SPA-211 crash-safety guarantee. After all writes succeed the dirty set is cleared.

pub fn hwm_for_label(&self, label_id: u32) -> Result<u64>

Return the high-water mark (slot count) for a label.

Returns 0 if no nodes have been created for that label yet.

pub fn col_ids_for_label(&self, label_id: u32) -> Result<Vec<u32>>

Discover all column IDs that currently exist on disk for label_id.

Scans the label directory for col_{id}.bin files and returns the parsed col_id values. Used by create_node to zero-pad columns that are not supplied for a new node (SPA-187).

Returns Err when the directory exists but cannot be read (e.g. permissions failure or I/O error). A missing directory is not an error — it simply means no nodes of this label have been created yet.

pub fn disk_hwm_for_label(&self, label_id: u32) -> Result<u64>

Return the on-disk high-water mark for a label, bypassing any in-memory advances made by peek_next_slot.

Used by [WriteTx::merge_node] to limit the disk scan to only slots that have actually been persisted.

pub fn peek_next_slot(&mut self, label_id: u32) -> Result<u32>

Reserve the slot index that the next create_node call will use for label_id, advancing the in-memory HWM so that the slot is not assigned again within the same NodeStore instance.

This is used by [WriteTx::create_node] to pre-compute a NodeId before the actual disk write, so the ID can be returned to the caller while the write is deferred until commit (SPA-181).

The on-disk HWM is not updated here; it is updated when the buffered NodeCreate operation is applied in commit().

pub fn create_node_at_slot( &mut self, label_id: u32, slot: u32, props: &[(u32, Value)], ) -> Result<NodeId>

Write a node at a pre-reserved slot (SPA-181 commit path).

Like [create_node] but uses the caller-specified slot index instead of deriving it from the HWM. Used by [WriteTx::commit] to flush buffered node-create operations in the exact order they were issued, with slots that were already pre-allocated by [peek_next_slot].

Advances the on-disk HWM to slot + 1 (or higher if already past that).

pub fn batch_write_node_creates( &mut self, writes: Vec<(u32, u32, u32, u64, bool)>, node_slots: &[(u32, u32)], ) -> Result<()>

Batch-write column data for multiple nodes created in a single transaction commit (SPA-212 write-amplification fix).

Why this exists

The naive path calls create_node_at_slot per node, which opens and closes every column file once per node. For a transaction that creates N nodes each with C columns, that is O(N × C) file-open/close syscalls.

This method instead:

1. Accepts pre-encoded (label_id, col_id, slot, raw_value, is_present) tuples from the caller (value encoding happens in commit() before the call).
2. Sorts by (label_id, col_id) so all writes to the same column file are contiguous.
3. Opens each (label_id, col_id) file exactly once, writes all slots for that column, then closes it — reducing file opens to O(labels × cols).
4. Updates each null-bitmap file once per (label_id, col_id) group.

HWM advances are applied for every (label_id, slot) in node_slots, exactly as create_node_at_slot would do them. node_slots must include all created nodes — including those with zero properties — so that the HWM is advanced even for property-less nodes.

Rollback

On I/O failure the method truncates every file that was opened back to its pre-call size, matching the rollback contract of create_node_at_slot.

pub fn create_node( &mut self, label_id: u32, props: &[(u32, Value)], ) -> Result<NodeId>

Create a new node in label_id with the given properties.

Returns the new NodeId packed as (label_id << 32) | slot.

Slot alignment guarantee (SPA-187)

Every column file for label_id must have exactly node_count * 8 bytes so that slot N always refers to node N across all columns. When a node is created without a value for an already-known column, that column file is zero-padded to (slot + 1) * 8 bytes. The zero sentinel is recognised by read_col_slot_nullable as “absent” and surfaces as Value::Null in query results.

pub fn tombstone_node(&self, node_id: NodeId) -> Result<()>

Write a deletion tombstone (u64::MAX) into col_0.bin for node_id.

Creates col_0.bin (and its parent directory) if it does not exist, zero-padding all preceding slots. This ensures that nodes which were created without any col_0 property are still properly marked as deleted and become invisible to subsequent scans.

Called from [WriteTx::commit] when flushing a buffered NodeDelete.

pub fn set_node_col( &self, node_id: NodeId, col_id: u32, value: &Value, ) -> Result<()>

Overwrite the value of a single column for an existing node.

Seeks to the slot’s offset within col_{col_id}.bin and writes the new 8-byte little-endian value in-place. Returns Err(NotFound) if the slot does not exist yet.

pub fn upsert_node_col( &self, node_id: NodeId, col_id: u32, value: &Value, ) -> Result<()>

Write or create a column value for a node, creating and zero-padding the column file if it does not yet exist.

Unlike [set_node_col], this method creates the column file and fills all slots from 0 to slot - 1 with zeros before writing the target value. This supports adding new property columns to existing nodes (Phase 7 set_property semantics).

pub fn get_node_raw( &self, node_id: NodeId, col_ids: &[u32], ) -> Result<Vec<(u32, u64)>>

Retrieve all stored properties of a node.

Returns (col_id, raw_u64) pairs in the order the columns were defined. The caller knows the schema (col IDs) from the catalog.

pub fn get_node_raw_nullable( &self, node_id: NodeId, col_ids: &[u32], ) -> Result<Vec<(u32, Option<u64>)>>

Like [get_node_raw] but treats absent columns as None rather than propagating Error::NotFound.

A column is considered absent when:

• Its column file does not exist (property never written for the label).
• Its column file is shorter than slot + 1 entries (sparse write — an earlier node never wrote this column; a later node that did write it padded the file, but this slot’s value was never explicitly stored).

This is the correct read path for IS NULL evaluation: absent properties must appear as Value::Null, not as an error or as integer 0.

pub fn get_node( &self, node_id: NodeId, col_ids: &[u32], ) -> Result<Vec<(u32, Value)>>

Retrieve the typed property values for a node.

Convenience wrapper over [get_node_raw] that decodes every raw u64 back to a Value, reading the overflow string heap when needed (SPA-212).

pub fn read_col_all(&self, label_id: u32, col_id: u32) -> Result<Vec<u64>>

Read the entire contents of col_{col_id}.bin for label_id as a flat Vec<u64>. Returns an empty vec when the file does not exist yet.

This is used by crate::property_index::PropertyIndex::build to scan all slot values in one fs::read call rather than one read_col_slot per node, making index construction O(n) rather than O(n * syscall-overhead).

pub fn read_null_bitmap_all( &self, label_id: u32, col_id: u32, ) -> Result<Option<Vec<bool>>>

Read the null-bitmap for (label_id, col_id) and return a Vec<bool> where index i is true iff slot i has a real value (was explicitly written, as opposed to zero-padded for alignment or never written).

If no bitmap file exists (old data written before SPA-207), returns None to signal backward-compat mode — callers should fall back to the raw != 0 sentinel in that case.

Used by [PropertyIndex::build_for] and [PropertyIndex::build] to correctly index slots whose value is Int64(0) (raw = 0), which is otherwise indistinguishable from the absent sentinel without the bitmap.

pub fn batch_read_node_props_nullable( &self, label_id: u32, slots: &[u32], col_ids: &[u32], ) -> Result<Vec<Vec<Option<u64>>>>

Batch-read multiple slots from multiple columns, returning nullable results.

Like [batch_read_node_props] but preserves null semantics using the null-bitmap sidecar (col_{id}_null.bin). Each Option<u64> is:

• Some(raw) — the slot has a real stored value (may be 0 for Int64(0)).
• None — the slot was zero-padded, never written, or the null bitmap marks it absent.

Backward compatible: if no bitmap file exists for a column (data written before SPA-207), every slot whose column file entry is within bounds is treated as present (Some).

This replaces the raw-0-as-absent semantic of [batch_read_node_props] and is the correct API for the chunked pipeline (Phase 2, SPA-299).

pub fn batch_read_node_props( &self, label_id: u32, slots: &[u32], col_ids: &[u32], ) -> Result<Vec<Vec<u64>>>

Batch-read multiple slots from multiple columns.

Chooses between two strategies based on the K/N ratio:

• Sorted-slot reads (SPA-200): when K ≪ N, seeks to each slot offset — O(K × C) I/O instead of O(N × C). Slots are sorted ascending before the read so seeks are sequential.
• Full-column load: when K is close to N (>50% of column), reading the whole file is cheaper than many random seeks.

All slots must belong to label_id. Returns a Vec indexed parallel to slots; inner Vec indexed parallel to col_ids. Missing/out-of-range slots return 0.