nusy-graph-query 0.15.2

Graph-native semantic search for Arrow RecordBatches — embeddings, traversal, hybrid ranking, and caching
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134

# nusy-graph-query

[![Crates.io](https://img.shields.io/crates/v/nusy-graph-query.svg)](https://crates.io/crates/nusy-graph-query)
[![docs.rs](https://docs.rs/nusy-graph-query/badge.svg)](https://docs.rs/nusy-graph-query)
[![MIT License](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)

**Graph-native semantic search for Arrow RecordBatches** — embeddings,
traversal, hybrid ranking, and caching.

`nusy-graph-query` provides the building blocks for semantic search and graph
traversal over [Apache Arrow](https://arrow.apache.org/) data. It's designed
for knowledge graphs stored as RecordBatches, where you need to combine
structural graph queries with semantic similarity search.

## Features

- **`EmbeddingProvider` trait** — pluggable embedding backends (hash-based
  deterministic, Ollama API, subprocess sentence-transformers)
- **Graph traversal** — generic BFS/DFS over Arrow edge tables, parameterized
  by column indices via `EdgeSchema`
- **Hybrid ranking** — combine structural graph scores with semantic similarity
  using configurable weights
- **Embedding cache** — content-hash invalidation with Parquet persistence,
  so embeddings survive restarts without recomputation
- **Zero-copy Arrow** — operates directly on `RecordBatch` columns, no
  intermediate materialization

## Quick Start

```rust
use nusy_graph_query::{
    HashEmbeddingProvider, EmbeddingProvider,
    cosine_similarity, semantic_search,
    EmbeddedItem,
};

// Create a deterministic embedding provider (for testing or small datasets)
let provider = HashEmbeddingProvider::new(384);

// Embed some text
let vectors = provider.embed_batch(&[
    "Alice knows Bob".to_string(),
    "Cat is an animal".to_string(),
]).unwrap();

// Compute similarity
let sim = cosine_similarity(&vectors[0], &vectors[1]);
println!("Similarity: {sim:.4}");
```

### Graph Traversal

```rust
use nusy_graph_query::traversal::*;
use arrow::array::{RecordBatch, StringArray};

// Define your edge schema (which columns hold source/target/predicate)
let schema = EdgeSchema {
    source_col: 0,
    target_col: 1,
    predicate_col: Some(2),
};

// BFS from a node, following "calls" edges up to depth 3
let reachable = bfs("main", &edges_batch, &schema, Direction::Forward, Some("calls"), 3);
for node in &reachable {
    println!("  {} (depth {})", node.id, node.depth);
}
```

### Hybrid Ranking

```rust
use nusy_graph_query::{hybrid_rank, HybridConfig, RankCandidate};

let config = HybridConfig {
    structural_weight: 0.6,
    semantic_weight: 0.4,
};

// Combine structural graph scores with semantic similarity
let results = hybrid_rank(&candidates, &embeddings, "search query", &provider, &config, 10)?;
```

## Installation

```toml
[dependencies]
nusy-graph-query = "0.14"
```

### Feature Flags

| Flag | Default | Description |
|------|---------|-------------|
| `subprocess` | off | Python sentence-transformers provider (requires Python + sentence-transformers) |
| `fastembed` | off | Local ONNX embedding via fastembed-rs (~2ms/chunk, no network) |

## Architecture

```
nusy-graph-query
  embedding.rs          — EmbeddingProvider trait, hash provider, cosine similarity
  traversal.rs          — BFS/DFS over Arrow edge RecordBatches
  hybrid_rank.rs        — Weighted structural + semantic scoring
  cache.rs              — Content-hash embedding cache (Parquet persistence)
  fastembed_provider.rs — Local ONNX provider (feature: fastembed)
  subprocess.rs         — Python subprocess provider (feature: subprocess)
```

The crate operates on standard Apache Arrow `RecordBatch` data. Graph edges
can be stored as either:

- **Edge tables** — separate RecordBatch with source/target/predicate columns
  (use `build_adjacency` + `bfs`)
- **List columns** — dependencies stored as `List<Utf8>` on each node
  (use `build_adjacency_from_list` + `bfs_with_adjacency`)

## Minimum Supported Rust Version

Rust 2024 edition (1.85+). Uses `let-else` and `let chains`.

## Part of the NuSy Ecosystem

This crate is part of [nusy-product-team](https://github.com/hankh95/nusy-product-team),
a neurosymbolic AI platform. Related crates:

- **nusy-arrow-core** — Arrow schemas, Triple type, Namespace/YLayer enums
- **nusy-arrow-git** — Graph-native git operations on Arrow tables
- **nusy-dual-store** — Fast/slow dual-store with consolidation

## License

MIT