lore-engine 0.1.0

Pure Rust wiki engine — SQLite FTS5, petgraph link graph, wiki link parsing, file watching. No GUI dependencies.
Documentation

lore-engine

Pure Rust wiki engine. SQLite FTS5 full-text search, petgraph link graph, wiki link parsing, file watching. No GUI dependencies.

What it is

A framework-agnostic engine for building personal wiki / knowledge base applications. You provide a folder of Markdown files; lore-engine gives you:

  • Page CRUD with automatic wiki link resolution
  • Full-text search via SQLite FTS5 with BM25 ranking
  • Link graph — backlinks, forward links, orphan detection
  • Wiki link parsing — [[Page Name]] and [[Page Name|alias]] with pulldown-cmark code detection
  • Incremental sync — only processes files that changed (content-hash diffing)
  • File watching — optional, picks up external edits in real time
  • Placeholder pages — unresolved links create placeholders automatically

Who it's for

Anyone building a wiki, note-taking app, or knowledge management tool in Rust. The engine handles storage, search, and graph logic so you can focus on the UI.

Currently used by:

  • Lore GUI — Qt 6.5 desktop app (personal wiki)
  • lore-cli — command-line interface
  • lore-mcp — MCP server (exposes the wiki to AI assistants)

Install

[dependencies]

lore-engine = "0.1"

Quick start

use lore_engine::state::AppState;
use lore_engine::engine::vault_ops;
use std::path::Path;

let state = AppState::new();

// Open a vault (folder of .md files)
let info = vault_ops::open_vault(Path::new("./my-wiki"), &state).unwrap();
println!("{} pages, {} links", info.page_count, info.link_count);

// Create a page
vault_ops::create_page("My First Page", "", &state).unwrap();

// Save content (upserts — creates if doesn't exist)
vault_ops::save_page("my-first-page", "# My First Page\n\nSee [[Other Page]].\n", &state).unwrap();

// Load a page
let page = vault_ops::load_page("my-first-page", &state).unwrap();
println!("Title: {}, Backlinks: {:?}", page.title, page.backlinks);

// Search
let results = vault_ops::search_vault("first", 10, &state).unwrap();
for r in results {
    println!("{}: {}", r.slug, r.snippet);
}

Features

Feature What
Page CRUD Create, read, update, delete with automatic link resolution
Upsert save_page creates pages that don't exist, promotes placeholders
Wiki links [[Page]], [[Page|alias]], fuzzy slug resolution, .md extension stripping
Full-text search FTS5 with BM25 ranking, prefix matching, highlighted snippets
Title search Fast LIKE-based title/slug search for quick switchers
Link graph Directed graph (petgraph), backlinks, forward links, degree
Placeholders Unresolved links create placeholder nodes, cleaned up automatically
Incremental sync SHA-256 content hashing, only reprocesses changed files
File watching Optional debounced watcher for external edits (500ms)
Folder tree Build sidebar-style folder hierarchy from flat page list
Graph layout Force-directed layout for visualization
Settings JSON persistence for app settings (window geometry, theme, etc.)
H1 title extraction Page title derived from first # Heading in content
Filename sanitization Safe filenames on all platforms including Windows
Path traversal protection Folder parameters validated against .. and absolute paths

Architecture

AppState (Arc-friendly, Mutex-wrapped)
├── db: SQLite connection (FTS5, WAL mode, foreign keys)
├── vault_path: current vault folder
├── graph: in-memory WikiGraph (petgraph DiGraph)
└── watcher: optional file system debouncer

All operations go through vault_ops functions which coordinate DB, graph, filesystem, and FTS updates. The engine knows nothing about its consumers — any frontend (CLI, GUI, server) creates its own AppState and calls the same functions.

Known limitations

  • Wiki link syntax — supports [[Page]] and [[Page|alias]] only. Heading links ([[Page#Section]]) and embeds (![[Page]]) are not yet supported.
  • File watching — designed for desktop GUI use. The watcher holds locks during processing, which can cause contention under concurrent access. Lock ordering is consistent (no deadlock risk), but high-frequency concurrent writes may experience brief blocking.
  • Large files — content is loaded fully into memory. Files over ~10MB may cause high memory usage.
  • Schema migrations — the engine is on schema v1 with no migration ladder yet. Schema changes in future versions will require a migration mechanism.
  • Slug collisions — titles that differ only in punctuation (e.g. "C++" and "C#") may produce the same slug. The engine does not currently detect or prevent this.

License

Licensed under either of

at your option.