Skip to main content

Module pipeline

Module pipeline 

Source
Expand description

Data processing pipelines Pipeline Module

This module provides data pipeline capabilities for GraphRAG:

  • Data import from multiple formats
  • ETL (Extract, Transform, Load) operations
  • Batch and streaming processing

§Features

§Data Import

  • CSV/TSV file support
  • JSON/JSONL support
  • RDF/Turtle semantic web formats
  • GraphML graph exchange format
  • Streaming ingestion

§Validation

  • Schema validation
  • Data quality checks
  • Error handling and reporting

Re-exports§

pub use data_import::DataFormat;
pub use data_import::ImportConfig;
pub use data_import::ColumnMappings;
pub use data_import::ImportedEntity;
pub use data_import::ImportedRelationship;
pub use data_import::ImportResult;
pub use data_import::ImportError;
pub use data_import::DataImporter;
pub use data_import::StreamingImporter;
pub use data_import::StreamingSource;

Modules§

data_import
Data Import Pipeline