Skip to main content

Module writer

Module writer 

Source
Expand description

GraphWriter — buffered Parquet write path for the UUID-first topology / properties layout (#579).

Callers mint UUIDv7 identifiers (via graphforge_core::uuid::new_v7) and feed nodes, edges, and properties to the writer; it assigns integer surrogate IDs (node_id / edge_id), buffers rows in memory, and materialises them to Parquet on flush. The output round-trips through GraphCatalog.

Routing depends on OntologyMode:

edgesnode properties
Strict / Advisorytopology/edges/TYPENAME.parquet (TYPED_EDGE_SCHEMA)properties/TYPENAME.parquet
Exploratorytopology/edges/_exploratory.parquet (EXPLORATORY_EDGE_SCHEMA)properties/_untyped.parquet

Edge properties (#784) are written separately under edge_properties/REL_TYPE.parquet, keyed by edge_uuid and routed by relation name in every mode (a dedicated directory so a relation type cannot collide with a node label of the same name in properties/).

§Behaviour and limitations (baseline write path)

  1. flush merges the buffered rows with whatever is already on disk (read-modify-write), so separate write sessions accumulate (#733). Each file write is atomic (temp + rename, #790) — an I/O failure mid-write leaves the prior file intact — but the merge is per file: a failure between files commits some files and not others (nodes first, so the partial state is consistent; durability/fsync stays out of scope for this non-production, small-graph engine). There is no cross-session dedup: pure CREATE mints fresh UUIDs, so a node_uuid never recurs; MATCH…CREATE upsert is deferred to #703.
  2. Surrogate node_id / edge_id values start at 1 (0 is reserved as a sentinel) and continue from the on-disk maximum when a writer is opened on an existing project, so appended rows get fresh, monotonic surrogates.
  3. _untyped property schemas are inferred from the buffered literals (union of property names, type from the first non-null value seen). A column that sees conflicting scalar types uses a tagged scalar struct so values retain their openCypher types.
  4. In Advisory / Strict mode the writer trusts the caller’s rel_type as the typed edge file name — it performs no ontology validation here (that lives in the execution layer, which holds the ontology handle).
  5. _untyped property files are not auto-registered by [GraphCatalog] yet; only node_uuid is in its read schema until the runtime catalog learns the columns.
  6. The writer only ever creates topology/ and properties/ (the always-on baseline capabilities); capability-gated directories for other features are deferred to when those capabilities exist.

Structs§

GraphWriter
Buffered Parquet writer for graph topology and properties.

Functions§

count_entity_properties
Count non-null properties owned by the selected persisted entities across every dynamic-schema property partition.
read_entity_properties
Read the complete non-null property map for one persisted entity.
read_entity_property_keys
Read the non-null property keys currently stored for one entity.
read_node_property_rows
Read every UUID-keyed node property row from one persisted stem.
remove_edge_properties
Rewrite edge_properties/<rel_stem>.parquet applying per-edge_uuid REMOVE removals, staged and committed as one batch (#790).
remove_node_properties
Rewrite properties/<stem>.parquet applying per-node_uuid REMOVE removals, staged and committed as one batch (#790).
set_edge_properties_rewrite
Rewrite edge_properties/<rel_stem>.parquet applying per-edge_uuid SET updates, staged and committed as one batch (#790).
set_node_properties
Rewrite properties/<stem>.parquet applying per-node_uuid SET updates, staged and committed as one batch (#790).
stage_remove_edge_properties
Stage a rewrite of edge_properties/<rel_stem>.parquet applying per-edge_uuid REMOVE removals. Edge analogue of stage_remove_node_properties.
stage_remove_node_properties
Stage a rewrite of properties/<stem>.parquet applying per-node_uuid REMOVE removals. Returns the number of distinct nodes targeted.
stage_set_edge_properties
Stage a rewrite of edge_properties/<rel_stem>.parquet applying per-edge_uuid SET updates. Edge analogue of stage_set_node_properties; the join key is edge_uuid and the file is routed by relation name.
stage_set_node_properties
Stage a rewrite of properties/<stem>.parquet applying per-node_uuid SET updates into staged (committed by the caller, #790).

Type Aliases§

PendingNodeMatch
Identity, labels, and properties of a buffered node matched by MERGE.