graphrag-core 0.2.0

Core portable library for GraphRAG - works on native and WASM
Documentation
# GraphRAG System Architecture

This document describes the architecture of the GraphRAG (Graph Retrieval Augmented Generation) system.

## Overview

GraphRAG is a hybrid system that combines knowledge graphs with large language models (LLMs) to provide accurate and contextually relevant responses to user queries.

### System Components

The system consists of several interconnected modules:

#### 1. Document Processing Pipeline

The document processing pipeline handles ingestion and analysis of input documents:

- **Text Chunking**: Splits documents into manageable segments
- **Entity Extraction**: Identifies entities using NLP techniques
- **Relationship Detection**: Discovers connections between entities

#### 2. Knowledge Graph Storage

The knowledge graph stores extracted information:

- **Entities**: Nodes representing real-world objects
- **Relationships**: Edges connecting related entities
- **Metadata**: Additional context and provenance

#### 3. Retrieval System

The retrieval system implements multiple strategies:

##### Semantic Retrieval

Uses vector embeddings to find semantically similar content. This approach captures meaning beyond exact keyword matches.

##### Keyword Retrieval (BM25)

Traditional keyword-based search using BM25 ranking algorithm. Effective for queries with specific terminology.

##### Hybrid Retrieval

Combines semantic and keyword approaches using rank fusion techniques for optimal results.

### Query Processing

When a user submits a query, the system:

1. Analyzes the query to extract intent and entities
2. Retrieves relevant subgraphs from the knowledge base
3. Ranks and filters results based on relevance
4. Generates a response using the LLM with retrieved context

## Implementation Details

### Technology Stack

- **Language**: Rust for performance and safety
- **Graph Structure**: Custom in-memory graph implementation
- **Embeddings**: Support for multiple providers (OpenAI, Voyage, local models)
- **LLM Integration**: Compatible with various language models

### Performance Characteristics

The system is optimized for:
- Low-latency retrieval (< 100ms for typical queries)
- Efficient memory usage through streaming processing
- Scalability to large document collections

## Conclusion

The GraphRAG architecture provides a robust foundation for building knowledge-intensive applications that require both structured reasoning and natural language generation capabilities.