Skip to main content

Crate graphforge_storage

Crate graphforge_storage 

Source
Expand description

GraphForge StorageProvider trait, canonical Arrow schemas, and Parquet backend.

  • schemas — Arrow schema constants for every Parquet file
  • catalog — DataFusion TableProvider / CatalogProvider implementations (#572)
  • writer — buffered Parquet write path (GraphWriter) (#579)
  • mutator — in-place rewrite primitives for DELETE/DETACH DELETE (#740)
  • staging — temp-file + atomic-rename Parquet commit (RewriteBatch) (#790)
  • adjacency — on-disk CSR format for the derived adjacency index (#758, ADR 0005)
  • generation — topology_generation counter, the staleness signal for derived indexes (#759)
  • search_manifest / search_publication — shared M19 search freshness and atomic publication

Re-exports§

pub use generation::commit_topology_aware;
pub use generation::read_search_generation;
pub use generation::read_topology_generation;
pub use generation::touches_search_source;
pub use graph_projection::GraphProjectionSelection;
pub use graph_projection::GraphProjectionSummary;
pub use graph_projection::materialize_graph_projection;
pub use project_generation::CURRENT_FILE;
pub use project_generation::FORMAT_FILE;
pub use project_generation::PROJECT_FORMAT_BYTES;
pub use project_generation::ProjectCapabilityDescriptor;
pub use project_generation::ProjectParticipantDescriptor;
pub use project_generation::ProjectParticipantSnapshot;
pub use project_generation::ResolvedProjectGeneration;
pub use project_generation::open_or_initialize_project;
pub use project_generation::resolve_project_generation;
pub use project_checkpoints::CheckpointCreateRequest;
pub use project_checkpoints::CheckpointDeleteRequest;
pub use project_checkpoints::CheckpointReceipt;
pub use project_checkpoints::CheckpointRecord;
pub use project_checkpoints::CheckpointRevertRequest;
pub use project_checkpoints::create_checkpoint;
pub use project_checkpoints::delete_checkpoint;
pub use project_checkpoints::list_checkpoints;
pub use project_checkpoints::open_checkpoint_generation;
pub use project_checkpoints::revert_checkpoint;
pub use project_publication::ProjectCapability;
pub use project_publication::ProjectGenerationRequest;
pub use project_publication::ProjectParticipant;
pub use project_publication::ProjectParticipantEncoding;
pub use project_publication::ProjectPublicationReceipt;
pub use project_publication::ProjectStageOutcome;
pub use project_publication::StagedParticipant;
pub use project_publication::StagedProjectGeneration;
pub use project_publication::ValidatedProjectGeneration;
pub use project_publication::published_project_transaction;
pub use project_publication::stage_project_generation;
pub use project_publication::stage_project_generation_optimistic;
pub use project_recovery::ProjectRecoveryReport;
pub use project_recovery::recover_project_transactions;
pub use project_portable::PortableExportReceipt;
pub use project_portable::PortableImportReceipt;
pub use project_portable::PortableProjectLimits;
pub use project_portable::encode_portable_project;
pub use project_portable::export_portable_project;
pub use project_portable::import_portable_project;
pub use project_portable::import_portable_project_file;
pub use workspace_participants::MAX_WORKSPACE_REPOSITORY_SNAPSHOT_BYTES;
pub use workspace_participants::MAX_WORKSPACE_REPOSITORY_SNAPSHOT_ENTRIES;
pub use workspace_participants::MAX_WORKSPACE_REPOSITORY_SNAPSHOT_ID_BYTES;
pub use workspace_participants::WORKSPACE_CAPABILITY_ID;
pub use workspace_participants::WORKSPACE_CAPABILITY_VERSION;
pub use workspace_participants::WORKSPACE_CONFIGURATION_FAMILY;
pub use workspace_participants::WORKSPACE_ONTOLOGY_FAMILY;
pub use workspace_participants::WORKSPACE_REPOSITORY_SNAPSHOT_FAMILY;
pub use workspace_participants::WORKSPACE_REPOSITORY_SNAPSHOT_VERSION;
pub use workspace_participants::WorkspaceConfiguration;
pub use workspace_participants::WorkspaceOntology;
pub use workspace_participants::WorkspaceOntologyMode;
pub use workspace_participants::WorkspaceOntologySourceFormat;
pub use workspace_participants::WorkspaceRepositoryDefinitionDigest;
pub use workspace_participants::WorkspaceRepositoryGitProvenance;
pub use workspace_participants::WorkspaceRepositorySnapshot;
pub use workspace_participants::WorkspaceRepositorySourceDigest;
pub use workspace_participants::empty_workspace_participants;
pub use embedding_identity::ChunkingIdentity;
pub use embedding_identity::EmbeddingCompatibilityDescriptor;
pub use embedding_identity::EmbeddingCompatibilityId;
pub use embedding_identity::EmbeddingCompatibilityInput;
pub use embedding_identity::EmbeddingContentDigest;
pub use embedding_identity::EmbeddingDisplayName;
pub use embedding_identity::EmbeddingDistance;
pub use embedding_identity::EmbeddingGenerationId;
pub use embedding_identity::EmbeddingNormalization;
pub use embedding_identity::EmbeddingProducerIdentity;
pub use embedding_identity::EmbeddingSourceFingerprint;
pub use embedding_identity::EmbeddingValueType;
pub use embedding_identity::TokenCountClass;
pub use embedding_identity::TokenizerIdentity;
pub use embedding_catalog::EMBEDDING_SPACE_CATALOG_VERSION;
pub use embedding_catalog::EmbeddingSpaceCatalog;
pub use embedding_catalog::EmbeddingSpaceCatalogEntry;
pub use embedding_catalog::EmbeddingSpaceCatalogLimits;
pub use embedding_catalog::EmbeddingSpaceCatalogUpdate;
pub use embedding_catalog::bind_existing_embedding_space_catalog_entry;
pub use embedding_catalog::read_embedding_space_catalog;
pub use embedding_catalog::remove_embedding_space_catalog_identity;
pub use embedding_catalog::update_embedding_space_catalog;
pub use embedding_discovery::DiscoveredEmbeddingSpace;
pub use embedding_discovery::EmbeddingSpaceDiscoveryLimits;
pub use embedding_discovery::MAX_DISCOVERED_EMBEDDING_DESCRIPTOR_BYTES;
pub use embedding_discovery::MAX_DISCOVERED_EMBEDDING_SPACES;
pub use embedding_discovery::MAX_EMBEDDING_SPACE_DIRECTORY_ENTRIES;
pub use embedding_discovery::discover_embedding_spaces;
pub use embedding_batch::EmbeddingBatchRow;
pub use embedding_batch::ValidatedEmbeddingBatch;
pub use embedding_batch::validate_embedding_batch;
pub use embedding_manifest::EMBEDDING_GENERATION_MANIFEST_VERSION;
pub use embedding_manifest::EmbeddingGenerationManifest;
pub use embedding_manifest::EmbeddingGenerationManifestInput;
pub use embedding_manifest::EmbeddingPublicationFingerprint;
pub use embedding_manifest::EmbeddingSourceState;
pub use embedding_manifest::MAX_EMBEDDING_GENERATION_MANIFEST_BYTES;
pub use embedding_publication::EmbeddingGenerationPublication;
pub use embedding_publication::EmbeddingPublicationOutcome;
pub use embedding_publication::EmbeddingPublicationRequest;
pub use embedding_publication::current_embedding_generation;
pub use embedding_publication::delete_embedding_space_lineage;
pub use embedding_publication::publish_embedding_generation;
pub use embedding_freshness::EMBEDDING_SUBSTANTIAL_CHANGED_PERCENT;
pub use embedding_freshness::EMBEDDING_SUBSTANTIAL_MUTATION_BATCHES;
pub use embedding_freshness::EmbeddingForcedStaleDiagnostic;
pub use embedding_freshness::EmbeddingFreshness;
pub use embedding_freshness::EmbeddingFreshnessReason;
pub use embedding_freshness::EmbeddingFreshnessState;
pub use embedding_freshness::EmbeddingMutationObservation;
pub use embedding_freshness::EmbeddingReadDecision;
pub use embedding_freshness::classify_embedding_freshness;
pub use embedding_freshness::decide_embedding_read;
pub use embedding_refresh_config::DEFAULT_EMBEDDING_REFRESH_DEBOUNCE;
pub use embedding_refresh_config::EMBEDDING_REFRESH_CONFIG_VERSION;
pub use embedding_refresh_config::EmbeddingRefreshConfig;
pub use embedding_refresh_config::EmbeddingRefreshConfigLimits;
pub use embedding_refresh_config::EmbeddingRefreshConfigUpdate;
pub use embedding_refresh_config::EmbeddingRefreshFailureClass;
pub use embedding_refresh_config::EmbeddingRefreshOutcomeRecord;
pub use embedding_refresh_config::EmbeddingRefreshOutcomeStatus;
pub use embedding_refresh_config::EmbeddingRefreshProjectPolicy;
pub use embedding_refresh_config::EmbeddingRefreshSpacePolicy;
pub use embedding_refresh_config::EmbeddingRefreshSpaceState;
pub use embedding_refresh_config::MAX_EMBEDDING_REFRESH_CONFIG_BYTES;
pub use embedding_refresh_config::MAX_EMBEDDING_REFRESH_CONFIG_ENTRIES;
pub use embedding_refresh_config::MAX_EMBEDDING_REFRESH_JOBS;
pub use embedding_refresh_config::ResolvedEmbeddingRefreshPolicy;
pub use embedding_refresh_config::read_embedding_refresh_config;
pub use embedding_refresh_config::update_embedding_refresh_config;
pub use embedding_mutations::EMBEDDING_MUTATION_JOURNAL_VERSION;
pub use embedding_mutations::EmbeddingMutationBatch;
pub use embedding_mutations::EmbeddingMutationJournal;
pub use embedding_mutations::EmbeddingMutationJournalLimits;
pub use embedding_mutations::merge_embedding_mutation_batch;
pub use embedding_mutations::read_embedding_mutation_journal;
pub use embedding_mutations::reset_embedding_mutation_journal;
pub use search_manifest::MAX_SEARCH_ARTIFACT_KEY_BYTES;
pub use search_manifest::MAX_SEARCH_MANIFEST_BYTES;
pub use search_manifest::MAX_SEARCH_SELECTOR_BYTES;
pub use search_manifest::SEARCH_MANIFEST_VERSION;
pub use search_manifest::SearchArtifactError;
pub use search_manifest::SearchArtifactKey;
pub use search_manifest::SearchIndexKind;
pub use search_manifest::SearchManifest;
pub use search_manifest::SearchSourcePart;
pub use search_manifest::SearchSourceSnapshot;
pub use search_manifest::canonical_source_fingerprint;
pub use search_publication::PublishedSearchArtifact;
pub use search_publication::SearchCoordinationLimits;
pub use search_publication::SearchPublicationMode;
pub use search_publication::SearchPublicationOutcome;
pub use search_publication::SearchPublicationPlan;
pub use search_publication::SearchUpdateBuild;
pub use search_publication::cleanup_abandoned_search_builds;
pub use search_publication::coordinate_search_publication;
pub use search_publication::coordinate_search_update;
pub use search_publication::current_search_artifact;
pub use vector_store::StoredVector;
pub use vector_store::VECTOR_BACKEND_VERSION;
pub use vector_store::VECTOR_CONTRACT_VERSION;
pub use vector_store::VECTOR_DATA_FILE;
pub use vector_store::VectorSearchHit;
pub use vector_store::VectorStoreLimits;
pub use vector_store::VectorUpsertChange;
pub use vector_store::apply_vector_upsert;
pub use vector_store::read_vector_snapshot;
pub use vector_store::search_published_vectors;
pub use vector_store::upsert_published_vector;
pub use vector_store::validate_published_vectors;
pub use vector_store::validate_vector;
pub use vector_store::vector_schema;
pub use vector_store::write_vector_snapshot;
pub use io_stats::IoSnapshot;
pub use io_stats::snapshot as io_snapshot;
pub use catalog::EdgePropertyTable;
pub use catalog::GraphCatalog;
pub use catalog::PropertyTable;
pub use catalog::TopologyNodeTable;
pub use catalog::TypedEdgeTable;
pub use catalog::UnionEdgeTable;
pub use catalog::list_edge_property_stems;
pub use catalog::list_property_stems;
pub use catalog::read_edge_properties;
pub use catalog::read_edges;
pub use catalog::read_edges_filtered;
pub use catalog::read_nodes;
pub use catalog::read_nodes_filtered;
pub use catalog::read_properties;
pub use schemas::ADJACENCY_CSR_SCHEMA;
pub use schemas::ADJACENCY_MANIFEST_SCHEMA;
pub use schemas::EDGE_PROPERTY_BASE_SCHEMA;
pub use schemas::EXPLORATORY_EDGE_SCHEMA;
pub use schemas::PROPERTY_BASE_SCHEMA;
pub use schemas::TOPOLOGY_NODES_SCHEMA;
pub use schemas::TYPED_EDGE_SCHEMA;
pub use schemas::property_schema;
pub use schemas::property_type_to_arrow;
pub use schemas::result_schema;
pub use writer::GraphWriter;
pub use writer::count_entity_properties;
pub use writer::read_entity_properties;
pub use writer::read_entity_property_keys;
pub use writer::read_node_property_rows;
pub use writer::remove_edge_properties;
pub use writer::remove_node_properties;
pub use writer::set_edge_properties_rewrite;
pub use writer::set_node_properties;
pub use writer::stage_remove_edge_properties;
pub use writer::stage_remove_node_properties;
pub use writer::stage_set_edge_properties;
pub use writer::stage_set_node_properties;
pub use mutator::delete_edges;
pub use mutator::delete_nodes;
pub use mutator::delete_nodes_and_edges;
pub use mutator::incident_edge_uuids;
pub use mutator::stage_add_node_labels;
pub use mutator::stage_delete_edges;
pub use mutator::stage_delete_nodes;
pub use mutator::stage_mutate_node_labels;
pub use staging::RewriteBatch;
pub use staging::remove_stale_temps;

Modules§

adjacency
On-disk format for the derived adjacency index (indexes/adjacency/, ADR 0005).
adjacency_delta
Incremental adjacency: per-commit delta segments (#765).
catalog
DataFusion TableProvider and CatalogProvider implementations.
embedding_batch
Complete, canonical UUID/vector batches for embedding generations.
embedding_catalog
Durable caller-facing names for compatibility-addressed embedding spaces.
embedding_discovery
Bounded discovery of compatibility-addressed embedding generations.
embedding_freshness
Deterministic freshness policy for verified embedding generations.
embedding_identity
Versioned, collision-safe identities for primary embedding generations.
embedding_manifest
Durable source-state and complete-generation metadata for embeddings.
embedding_mutations
Durable per-lineage mutation evidence for embedding freshness.
embedding_publication
Crash-safe publication and reopen validation for complete embedding generations.
embedding_refresh_config
Durable, content-free embedding refresh policy and terminal outcomes.
generation
Project mutation generations — staleness signals for derived indexes.
graph_projection
Deterministic graph-only workspace projections.
io_stats
Process-global I/O counters for storage reads and staged rewrite commits. The read counters cover read_edges, read_edges_filtered, and read_nodes.
mutator
In-place mutation primitives for DELETE / DETACH DELETE (#740).
project_checkpoints
Durable named references to verified immutable project generations.
project_generation
Resolution of the one committed immutable project generation.
project_portable
Deterministic, bounded portable project-generation envelopes.
project_publication
Atomic publication of complete immutable project generations.
project_recovery
Deterministic recovery for interrupted project-generation transactions.
schemas
Canonical Arrow schemas for every Parquet file GraphForge reads or writes.
search_manifest
Canonical identity, manifests, and freshness for M19 search artifacts.
search_publication
Coordinated, crash-safe publication for text and vector search artifacts.
staging
Staged Parquet writes — temp-file + atomic-rename commit (#790).
vector_store
UUID-keyed primary vector snapshots and deterministic exact cosine search.
workspace_participants
Canonical generation-managed workspace ontology and configuration records.
writer
GraphWriter — buffered Parquet write path for the UUID-first topology / properties layout (#579).

Structs§

ParquetProvider
Parquet-backed storage provider stub.
StorageRow
Minimal row type exchanged between the storage layer and the executor. Will be replaced by Arrow RecordBatch in Milestone 13.

Enums§

GfError
All errors produced by GraphForge.

Traits§

StorageProvider
Abstraction over different storage backends.