nusy-codegraph

Code as a live Arrow object graph — parse source code into structured Arrow RecordBatches for zero-copy querying, versioning via git-native graph operations, semantic search, and impact analysis.

Note: This crate depends on internal NuSy workspace crates (nusy-arrow-core, nusy-arrow-git, nusy-graph-query, noesis-ship) and cannot compile standalone from crates.io. It is designed for use within the NuSy monorepo or as a framework reference for building your own code graph system.

For AI Developers

This crate demonstrates a complete code-as-graph pipeline:

Source code → tree-sitter CST → CodeNodes + CodeEdges → Arrow RecordBatches
                                  ↓
                          Semantic search + impact analysis

If you want to build a similar system for your own language or toolchain, the key ideas are:

1. Use tree-sitter for language-accurate CST parsing (not regex)
2. Arrow RecordBatches for zero-copy columnar storage and Parquet persistence
3. CodeEdges (calls, imports, inheritance) as first-class graph citizens
4. Semantic search via embeddings over code nodes (fastembed or ollama)
5. Git-native versioning via nusy-arrow-git WAL + atomic rename

Features

Code Parsing

• Rust — full support: functions, structs, enums, impl blocks, macros, const, static
• Python — partial support: functions, classes, async def

Code Graph

• CodeNode — every function, class, module as a typed node with location and signature
• CodeEdge — calls, imports, inheritance, containment as typed directed edges
• Cross-file edge resolution via NameResolver
• Arrow schema for zero-copy columnar storage

Semantic Search

• Embed code nodes via fastembed or ollama
• Cosine similarity search over the embedded graph
• Natural-language queries via codegraph_query_objects

Git-Native Graph Operations

• codegraph_diff — what changed in the graph between commits?
• codegraph_merge — three-way graph merge with conflict detection
• smart_merge — semantic merge using AST-aware conflict resolution
• Impact analysis: which nodes are affected by a change?

MCP Tools

• nusy-mcp-bridge exposes 4 tools as an MCP server:
  • codegraph_query_objects — search the graph
  • codegraph_add_edge / codegraph_remove_edge — modify edges
  • codegraph_update_object — update node metadata

Binaries

Architecture

Data Model

CodeNode
  id:            String      # "module::function" path
  kind:          String      # function | class | module | method | ...
  language:      String      # rust | python
  file_path:     String
  start_line:    u32
  end_line:      u32
  name:          String
  signature:     String      # full type signature
  doc:           String     # docstring

CodeEdge
  source_id:     String
  predicate:     String      # calls | imports | inherits | contains | ...
  target_id:     String
  file_path:     String      # where the edge was observed

Schema (Arrow)

code_nodes_schema()  // subject, predicate, object, graph, version columns
code_edges_schema()  // source_id, predicate, target_id, file_path columns

Ingestion Pipeline

1. Discover   → find all .py / .rs files by language
2. Parse      → tree-sitter CST → CodeNodes (functions, classes, etc.)
3. Extract    → extract_edges() builds Contains/Imports/Inheritance edges
4. Resolve    → NameResolver (Python) or RustModuleResolver (Rust)
                maps unqualified names → full CodeNode IDs
5. Write      → Arrow RecordBatch → Parquet (WAL + atomic rename)
6. Embed      → optional: embed_nodes() for semantic search

NameResolver (edges.rs) — Python and cross-language edges. Maps short names and qualified names to CodeNode IDs using a HashMap. Used by extract_edges() for Python files and as a fallback for Rust.

RustModuleResolver (module_resolver.rs) — Rust-specific resolution. Handles use statements → specific CodeNodes (not just module-level edges), impl Trait for Type → ImplementsTrait edges, pub use → ReExports edges. Used by extract_cross_file_edges() for Rust files.

extract_call_edges (edges.rs) — call edge extraction. When rust-analyzer is available, uses extract_scip_call_edges() for compiler-quality call detection via SCIP. Falls back to text scanning (name( pattern matching) when SCIP is unavailable.

Known limitations of text scanning: misses indirect calls, dynamic dispatch, and calls via variables. Use rust-analyzer for production-accuracy.

Quick Start

# Ingest a Rust project into the graph
nusy-codegraph-ingest ./path/to/project --language rust

# Query the self-graph (project ingests itself)
nusy-codegraph-query "query_objects: semantic_search_term"

# Start service mode (NATS)
nusy-codegraph-service --nats nats://192.168.8.110:4222

# MCP bridge for AI agent integration
nusy-mcp-bridge --nats nats://192.168.8.110:4222

As a Library

use nusy_codegraph::{
    ingest::ingest_directory,
    schema::{CodeNode, CodeEdge},
    search::semantic_search,
    edges::extract_edges,
};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Ingest a directory
    let result = ingest_directory("./src", Language::Rust)?;

    // Search semantically
    let results = semantic_search("error handling in REST API", top = 5)?;

    // Find all callers of a function
    let callers = nusy_codegraph::search::callers("my_crate::handle_request")?;

    Ok(())
}

Semantic Diff

use nusy_codegraph::semantic_diff;

let diff = semantic_diff("HEAD~1", "HEAD")?;
println!("{}", diff.format_impact_report());

Ecosystem

nusy-codegraph is part of the NuSy Arrow workspace:

Performance

Memory usage: ~500MB for 10k-node graph. Larger graphs scale linearly with node count.

Known bottlenecks:

• tree-sitter parsing is single-threaded per file (use --jobs for parallelism)
• extract_call_edges text scanning is O(source_lines × callable_count) when SCIP is unavailable

Limitations

• Python support is partial — only functions and classes are extracted. Type hints, decorators, and complex control flow are not yet parsed.
• Only Rust and Python — no TypeScript, Go, or other languages.
• Not standalone — requires nusy-arrow-core, nusy-arrow-git, nusy-graph-query, and noesis-ship as path dependencies. A fully FOSS version would need to publish those crates separately or reimplement against public crate equivalents.
• No incremental parsing — re-ingestion is a full re-parse. Large codebases pay the full tree-sitter cost on every update.

License

MIT