lore-engine 0.1.0

# lore-engine


Pure Rust wiki engine. SQLite FTS5 full-text search, petgraph link graph, wiki link parsing, file watching. No GUI dependencies.

## What it is


A framework-agnostic engine for building personal wiki / knowledge base applications. You provide a folder of Markdown files; lore-engine gives you:

- **Page CRUD** with automatic wiki link resolution
- **Full-text search** via SQLite FTS5 with BM25 ranking
- **Link graph** — backlinks, forward links, orphan detection
- **Wiki link parsing** — `[[Page Name]]` and `[[Page Name|alias]]` with pulldown-cmark code detection
- **Incremental sync** — only processes files that changed (content-hash diffing)
- **File watching** — optional, picks up external edits in real time
- **Placeholder pages** — unresolved links create placeholders automatically

## Who it's for


Anyone building a wiki, note-taking app, or knowledge management tool in Rust. The engine handles storage, search, and graph logic so you can focus on the UI.

Currently used by:
- **Lore GUI** — Qt 6.5 desktop app (personal wiki)
- **lore-cli** — command-line interface
- **lore-mcp** — MCP server (exposes the wiki to AI assistants)

## Install


```toml
[dependencies]
lore-engine = "0.1"
```

## Quick start


```rust
use lore_engine::state::AppState;
use lore_engine::engine::vault_ops;
use std::path::Path;

let state = AppState::new();

// Open a vault (folder of .md files)
let info = vault_ops::open_vault(Path::new("./my-wiki"), &state).unwrap();
println!("{} pages, {} links", info.page_count, info.link_count);

// Create a page
vault_ops::create_page("My First Page", "", &state).unwrap();

// Save content (upserts — creates if doesn't exist)
vault_ops::save_page("my-first-page", "# My First Page\n\nSee [[Other Page]].\n", &state).unwrap();

// Load a page
let page = vault_ops::load_page("my-first-page", &state).unwrap();
println!("Title: {}, Backlinks: {:?}", page.title, page.backlinks);

// Search
let results = vault_ops::search_vault("first", 10, &state).unwrap();
for r in results {
    println!("{}: {}", r.slug, r.snippet);
}
```

## Features


| Feature | What |
|---------|------|
| Page CRUD | Create, read, update, delete with automatic link resolution |
| Upsert | `save_page` creates pages that don't exist, promotes placeholders |
| Wiki links | `[[Page]]`, `[[Page\|alias]]`, fuzzy slug resolution, `.md` extension stripping |
| Full-text search | FTS5 with BM25 ranking, prefix matching, highlighted snippets |
| Title search | Fast LIKE-based title/slug search for quick switchers |
| Link graph | Directed graph (petgraph), backlinks, forward links, degree |
| Placeholders | Unresolved links create placeholder nodes, cleaned up automatically |
| Incremental sync | SHA-256 content hashing, only reprocesses changed files |
| File watching | Optional debounced watcher for external edits (500ms) |
| Folder tree | Build sidebar-style folder hierarchy from flat page list |
| Graph layout | Force-directed layout for visualization |
| Settings | JSON persistence for app settings (window geometry, theme, etc.) |
| H1 title extraction | Page title derived from first `# Heading` in content |
| Filename sanitization | Safe filenames on all platforms including Windows |
| Path traversal protection | Folder parameters validated against `..` and absolute paths |

## Architecture


```
AppState (Arc-friendly, Mutex-wrapped)
├── db: SQLite connection (FTS5, WAL mode, foreign keys)
├── vault_path: current vault folder
├── graph: in-memory WikiGraph (petgraph DiGraph)
└── watcher: optional file system debouncer
```

All operations go through `vault_ops` functions which coordinate DB, graph, filesystem, and FTS updates. The engine knows nothing about its consumers — any frontend (CLI, GUI, server) creates its own `AppState` and calls the same functions.

## Known limitations


- **Wiki link syntax** — supports `[[Page]]` and `[[Page|alias]]` only. Heading links (`[[Page#Section]]`) and embeds (`![[Page]]`) are not yet supported.
- **File watching** — designed for desktop GUI use. The watcher holds locks during processing, which can cause contention under concurrent access. Lock ordering is consistent (no deadlock risk), but high-frequency concurrent writes may experience brief blocking.
- **Large files** — content is loaded fully into memory. Files over ~10MB may cause high memory usage.
- **Schema migrations** — the engine is on schema v1 with no migration ladder yet. Schema changes in future versions will require a migration mechanism.
- **Slug collisions** — titles that differ only in punctuation (e.g. "C++" and "C#") may produce the same slug. The engine does not currently detect or prevent this.

## License


Licensed under either of

- Apache License, Version 2.0 ([LICENSE-APACHE](LICENSE-APACHE) or <http://www.apache.org/licenses/LICENSE-2.0>)
- MIT license ([LICENSE-MIT](LICENSE-MIT) or <http://opensource.org/licenses/MIT>)

at your option.