grafeo_loro/lib.rs
1//! # grafeo-loro
2//!
3//! Local-first, in-process, dual-store graph database with CRDT consensus
4//! (Loro) + execution (Grafeo). Issue #1 compliance: granular feature gates,
5//! WASM target support, trait-abstracted runtime, tree-as-graph adapter.
6//!
7//! ## Feature matrix (issue #1 item 7)
8//!
9//! | Feature | Pulls | WASM-safe |
10//! |---------------|----------------------------------------|-----------|
11//! | `bridge` | `lorosurgeon`, `parking_lot` | yes |
12//! | `batcher` | `bridge` + `tokio` (sync, time) | no |
13//! | `compression` | `lz4_flex`, `zstd` (C lib) | no* |
14//! | `tree` | `bridge` | yes |
15//! | `storage` | `async-trait` | yes |
16//! | `grafeo` | `bridge` + `grafeo` (native only) | no |
17//! | `onnx` | `grafeo` + `ort` (native only) | no |
18//! | `webrtc` | `webrtc-rs` (native only) | no |
19//! | `telemetry` | `opentelemetry`, `tracing` | yes |
20//! | `wasm` | `wasm-bindgen`, `js-sys`, `web-sys` | required |
21//! | `parallel` | `rayon` | no |
22//! | `serde` | `serde`, `serde_json` | yes |
23//! | `full` | all native features | no |
24//!
25//! `*` `compression` uses `zstd` which binds to a C library; do NOT enable
26//! it in WASM builds. Use `lz4_flex` only via a future `compression-lz4`
27//! sub-feature, or disable compression entirely in the browser.
28//!
29//! ### Issue #4: `OfflineOpQueue` + `EpochTracker` (WASM-accessible)
30//!
31//! Issue #4 factored `OfflineOpQueue`, `LineageEpoch`, `EpochMismatchError`,
32//! and the new standalone `EpochTracker` out of `src/bridge/sync_engine.rs`
33//! into `src/bridge/queue.rs`. The new module is gated **only** by
34//! `feature = "bridge"` — no `batcher` (tokio), no `grafeo` (native
35//! ONNX/ort), no `telemetry` (opentelemetry native). Browser consumers on
36//! `wasm32-unknown-unknown` can name these types directly via
37//! `grafeo_loro::OfflineOpQueue` / `grafeo_loro::EpochTracker`.
38//!
39//! ### `bridge` queue + epoch tracker sub-API
40//!
41//! | Type | Feature gate | Purpose |
42//! |----------------------|--------------|------------------------------------------|
43//! | `OfflineOpQueue` | `bridge` | Cap-bounded FIFO of serialized LoroOps |
44//! | `LineageEpoch` | `bridge` | `u64` type alias for the epoch counter |
45//! | `EpochMismatchError` | `bridge` | Error raised on remote/local epoch drift |
46//! | `EpochTracker` | `bridge` | Standalone `Arc<AtomicU64>` epoch store |
47//!
48//! ## Onde's recommended feature set
49//!
50//! ```toml
51//! grafeo-loro = { version = "0.4", default-features = false, features = ["bridge", "batcher", "compression", "tree"] }
52//! ```
53//!
54//! ## Minimal WASM smoke test
55//!
56//! ```toml
57//! grafeo-loro = { version = "0.4", default-features = false, features = ["bridge", "tree", "wasm"] }
58//! ```
59//!
60//! ## MSRV
61//!
62//! Rust 1.80+ (verified via `cargo msrv` against the dep tree).
63//!
64//! ## Issue #3 sub-issue → feature mapping
65//!
66//! Issue #3 ("Browser WASM consumers Support") has 10 sub-issues. Their
67//! feature gates are:
68//!
69//! | Sub-issue | Topic | Feature gate |
70//! |-----------|------------------------------------|-------------------------------------------|
71//! | #1 | WASM target compile + binary size | `wasm` + `bridge,tree,compression` |
72//! | #2 | Trait-abstracted runtime (Mailbox) | `bridge` (trait); `batcher` (Tokio impl) |
73//! | #3 | LoroDoc ownership API | `bridge,storage,grafeo,batcher` |
74//! | #4 | Merge semantics | (no feature — use `doc.import()`) |
75//! | #5 | Offline op-queue + lineage epoch | `bridge` (factored out in issue #4) |
76//! | #6 | FTS + SAB + shadow commits | `fts`, `sab`, `shadow` |
77//! | #7 | Graph invariants (cycle, root) | `bridge,tree` |
78//! | #8 | Presence (ephemeral overlay) | `bridge` (always available with bridge) |
79//! | #9 | Storage backend trait | `storage` |
80//! | #10 | Observability hooks | `observability` |
81//!
82//! ## No `merge` / `awareness` / `persistence` feature
83//!
84//! If you saw a feature flag named `merge`, `awareness`, or `persistence`
85//! in a downstream consumer's Cargo.toml, it was invented — grafeo-loro
86//! has never declared these. Enabling them now produces a `compile_error!`
87//! pointing at the correct alternative:
88//!
89//! | Invented name | Correct alternative |
90//! |-----------------|----------------------------------------------------------------|
91//! | `merge` | `doc.import(other.export(loro::ExportFormat::Snapshot))` |
92//! | `awareness` | `presence` module (always available with `bridge`) |
93//! | `persistence` | `storage` feature (`StorageBackend` trait + `InMemoryStorage`) |
94//!
95//! ## WASM binary size
96//!
97//! With the full WASM-safe feature set (`bridge,tree,compression,wasm,serde,
98//! fts,sab,shadow,observability`) and `opt-level="z"` + `lto=true` +
99//! `codegen-units=1` + `strip=true`, the raw `.wasm` (before `wasm-opt -Oz`)
100//! is approximately **2.26 MB**. After `wasm-opt -Oz` (configured in
101//! `[package.metadata.wasm-pack.profile.release]`), it shrinks ~15-20%.
102//!
103//! For size-constrained consumers (e.g. <1.8 MB budget), drop `fts` (inverted
104//! index) and `shadow` (Git DAG shadow commits) first — they together add
105//! ~400 KB. The minimal smoke feature set `bridge,tree,wasm` produces a
106//! ~800 KB `.wasm` after `wasm-opt -Oz`.
107
108// ============================================================================
109// Always-on core: error, constants, config, types
110// ============================================================================
111// These modules have no heavy deps; they form the shared vocabulary of the
112// crate and are always available regardless of feature selection.
113
114// ============================================================================
115// Issue #4 secondary finding #2: fail-loud on invented feature names
116// ============================================================================
117//
118// Issue #4 reports that downstream consumers have been observed inventing
119// feature names like `merge`, `awareness`, `persistence` in their Cargo.toml
120// and shipping stubs referencing them. The crate has never declared these
121// features — older Cargo silently accepted unknown `--features` flags (no-op),
122// newer Cargo warns but does not error.
123//
124// To fail loud and clear, we declare them as empty feature stubs in
125// Cargo.toml and emit a `compile_error!` here pointing the consumer at the
126// correct alternative. This is a breaking change (consumers who relied on
127// the silent no-op behavior will now get a hard error) — intentional per
128// issue #4's "no backward compat" mandate.
129
130#[cfg(feature = "merge")]
131compile_error!(
132 "grafeo-loro has no `merge` feature. To merge two LoroDocs, call \
133 `doc.import(other_doc.export(loro::ExportFormat::Snapshot))` — see \
134 the Loro CRDT docs. The `merge` feature name was invented by a \
135 downstream consumer stub; remove it from your Cargo.toml."
136);
137
138#[cfg(feature = "awareness")]
139compile_error!(
140 "grafeo-loro has no `awareness` feature. Presence/awareness is provided \
141 by the `presence` module, which is always available when the `bridge` \
142 feature is enabled — no separate feature flag needed. The `awareness` \
143 feature name was invented by a downstream consumer stub; remove it \
144 from your Cargo.toml."
145);
146
147#[cfg(feature = "persistence")]
148compile_error!(
149 "grafeo-loro has no `persistence` feature. Persistence is provided by \
150 the `storage` feature (StorageBackend trait + InMemoryStorage reference \
151 impl). Enable `storage` in your Cargo.toml. The `persistence` feature \
152 name was invented by a downstream consumer stub; remove it."
153);
154
155pub mod config;
156pub mod constants;
157pub mod error;
158pub mod types;
159
160// ============================================================================
161// Feature-gated modules
162// ============================================================================
163
164/// Bridge: bidirectional Loro↔Grafeo sync types + id-mapping.
165///
166/// Required by `bridge` feature. The `MutationBatcher` requires `batcher`
167/// (which pulls `tokio`). The `SyncEngine` itself requires `batcher` because
168/// it owns the tokio MPSC channels.
169#[cfg(feature = "bridge")]
170pub mod bridge;
171
172/// Compression envelope + LoroDoc extension trait.
173///
174/// Required by `compression` feature. Pulls `lz4_flex` + `zstd`.
175#[cfg(feature = "compression")]
176pub mod compression;
177
178/// Hydration: cold-boot rebuild of Grafeo indexes from a Loro snapshot.
179///
180/// `parallel_hydrate_grafeo` requires the `parallel` feature (pulls `rayon`).
181/// The serial fallback is always available when `grafeo` is on.
182#[cfg(all(feature = "grafeo", not(target_family = "wasm")))]
183pub mod hydration;
184
185/// Ephemeral presence: WebSocket-channel %EPH envelope.
186///
187/// Independent of bridge/grafeo. `webrtc` feature wires a transport.
188#[cfg(feature = "bridge")]
189pub mod presence;
190
191/// Grafeo schema entities: `VertexEntity`, `EdgeEntity`, `OrderedCollection`.
192#[cfg(feature = "bridge")]
193pub mod schema;
194
195/// Storage backend trait + in-memory reference impl.
196#[cfg(feature = "storage")]
197pub mod storage;
198
199/// Telemetry: metrics, traces, health probes.
200#[cfg(feature = "telemetry")]
201pub mod telemetry;
202
203/// Tree-as-graph adapter (issue #1 item 8).
204///
205/// First-class `tree` module that turns `:CHILD` edges into ergonomic
206/// `parent` / `children` / `indent` / `outdent` operations. Independent
207/// of grafeo internals — works against the bridge surface only.
208#[cfg(feature = "tree")]
209pub mod tree_adapter;
210
211/// FFI-friendly hot-path API (issue #1 item 6).
212///
213/// `NodeOp` is `#[repr(C)]` using `&str` not `String`. Bincode-only
214/// `apply_loro_op_bytes` for sub-µs FFI.
215#[cfg(feature = "bridge")]
216pub mod ffi;
217
218/// WASM bindings: `JsValue` error bridge + `wasm-bindgen` prelude (issue #1
219/// item 12).
220///
221/// The module itself is gated by `feature = "wasm"` only — the target-
222/// agnostic `error_code` function is available on native (for testing) too.
223/// The JsValue-using pieces (`js_error`, `From<GrafeoLoroError> for JsValue`,
224/// `init_panic_hook`) are internally gated by `target_family = "wasm"` so
225/// they only compile on actual WASM targets.
226#[cfg(feature = "wasm")]
227pub mod wasm;
228
229/// Trait-abstracted async runtime (issue #1 item 2).
230///
231/// `Mailbox<T>` trait lets Onde plug in `wasm-bindgen-futures` +
232/// `web-sys::MessageChannel` instead of `tokio::sync::mpsc`. The tokio
233/// impl lives behind the `batcher` feature.
234#[cfg(feature = "bridge")]
235pub mod runtime;
236
237// ============================================================================
238// Issue #3 sub-issue 6: native shadow commits, FTS, SAB layout
239// ============================================================================
240
241/// Native Git DAG / shadow commit API (issue #3 sub-issue 6).
242///
243/// Exposes per-writer WIP ref namespace (`refs/wip/<peerId>`) so downstream
244/// no longer needs `isomorphic-git` (80KB) for bounded undo.
245#[cfg(feature = "shadow")]
246pub mod shadow;
247
248/// Lightweight native full-text-search inverted index (issue #3 sub-issue 6).
249///
250/// <20MB WASM memory ceiling. Replaces heavy ONNX/HNSW runtimes for
251/// text-search workloads.
252#[cfg(feature = "fts")]
253pub mod fts;
254
255/// Native SAB (SharedArrayBuffer) layout writer (issue #3 sub-issue 6).
256///
257/// Pushes Y-offset / height math to Rust and writes directly to a SAB
258/// pointer so the JS virtualizer doesn't have to.
259#[cfg(feature = "sab")]
260pub mod sab;
261
262// ============================================================================
263// Issue #3 sub-issue 10: observability
264// ============================================================================
265
266/// Observability hooks: queue state, fault injection, invariant checks
267/// (issue #3 sub-issue 10).
268///
269/// Exposes internal queue state (`depth`, `oldest_age`, `locked_nodes`) to
270/// JS. Adds fault-injection hooks for testing. Provides invariant-check
271/// API for I4/I5/I11/I12/I14 post-mutation assertions.
272#[cfg(feature = "observability")]
273pub mod observability;
274
275// ============================================================================
276// Top-level facade: `GrafeoLoroApp`
277// ============================================================================
278//
279// The app facade requires `bridge` + `storage` + `telemetry` to be useful
280// in production. Tests that exercise the facade enable `full`.
281
282#[cfg(all(
283 feature = "bridge",
284 feature = "storage",
285 feature = "grafeo",
286 feature = "batcher"
287))]
288pub mod app;
289
290// ============================================================================
291// Re-exports
292// ============================================================================
293
294#[cfg(all(
295 feature = "bridge",
296 feature = "storage",
297 feature = "grafeo",
298 feature = "batcher"
299))]
300pub use app::GrafeoLoroApp;
301#[cfg(all(feature = "batcher", feature = "grafeo", feature = "telemetry"))]
302pub use bridge::sync_engine::InboundMsg;
303#[cfg(feature = "bridge")]
304pub use bridge::BridgeMaps;
305#[cfg(all(feature = "batcher", feature = "grafeo", feature = "telemetry"))]
306pub use bridge::SyncEngine;
307
308// Issue #4: OfflineOpQueue + EpochTracker are reachable from WASM with
309// `feature = "bridge"` only (no batcher/grafeo/telemetry needed).
310// Factored out of `src/bridge/sync_engine.rs` in issue #4 so browser
311// consumers on `wasm32-unknown-unknown` can name these types directly
312// from the crate root.
313#[cfg(feature = "bridge")]
314pub use bridge::queue::{EpochMismatchError, EpochTracker, LineageEpoch, OfflineOpQueue};
315
316// Issue #4: #[wasm_bindgen] JS-facing wrappers. Available only on actual
317// WASM targets (the wrappers themselves are `#![cfg(target_family = "wasm")]`
318// inside src/wasm/queue.rs).
319#[cfg(all(feature = "wasm", target_family = "wasm"))]
320pub use wasm::queue::{WasmEpochTracker, WasmOfflineOpQueue};
321
322#[cfg(feature = "compression")]
323pub use compression::{CompressedPayload, LoroDocCompressionExt};
324pub use config::{CompressionType, SsotMode};
325pub use error::GrafeoLoroError;
326pub use error::Result;
327#[cfg(all(feature = "grafeo", not(target_family = "wasm")))]
328pub use hydration::hydrate_grafeo;
329#[cfg(all(feature = "grafeo", feature = "parallel", not(target_family = "wasm")))]
330pub use hydration::parallel_hydrate_grafeo;
331#[cfg(all(feature = "grafeo", not(target_family = "wasm")))]
332pub use hydration::vector::generate_local_embedding;
333#[cfg(all(feature = "grafeo", not(target_family = "wasm")))]
334pub use hydration::VectorOffloadManager;
335#[cfg(feature = "storage")]
336pub use storage::InMemoryStorage;
337#[cfg(feature = "storage")]
338pub use storage::StorageBackend;
339// Trait-abstracted async runtime (issue #1 item 2). The `Mailbox` trait +
340// `MailboxClosed` error are available with `bridge` alone; the tokio-backed
341// `TokioMailbox` impl requires `batcher` (which pulls `tokio::sync::mpsc`).
342#[cfg(feature = "batcher")]
343pub use runtime::TokioMailbox;
344#[cfg(feature = "bridge")]
345pub use runtime::{Mailbox, MailboxClosed};
346// Re-exports for tree_adapter (issue #1 item 8). The `tree` feature gates
347// the module itself; the re-exports follow the same gate. `CycleError`
348// derives `thiserror::Error` (a non-optional dep), so it is always available
349// when `tree` is on.
350#[cfg(feature = "tree")]
351pub use tree_adapter::{CycleError, TreeAdapter, TreeNode};
352// Re-exports for ffi, wasm modules will be added by parallel agents (tasks
353// 6, 8) when they fill in their stub modules.
354
355// Re-exports for the FFI hot-path API (issue #1 item 6). `NodeOp` and
356// `NodeValue` are available with `bridge` alone (they have no grafeo deps);
357// `apply_node_batch` additionally needs `grafeo` (calls `apply_loro_op`);
358// `apply_loro_op_bytes` additionally needs `serde` (bincode 1.x requires
359// `LoroOp: Deserialize`, which is derived under `serde`).
360#[cfg(all(feature = "bridge", feature = "grafeo", feature = "serde"))]
361pub use ffi::apply_loro_op_bytes;
362#[cfg(all(feature = "bridge", feature = "grafeo"))]
363pub use ffi::apply_node_batch;
364#[cfg(feature = "bridge")]
365pub use ffi::{NodeOp, NodeValue};
366
367// Re-exports for the WASM JsValue error bridge (issue #1 item 12).
368// `error_code` is target-agnostic (testable on native); `js_error` +
369// `init_panic_hook` only exist on `target_family = "wasm"`. The
370// `From<GrafeoLoroError> for JsValue` impl is brought into scope by
371// `use crate::wasm::*` on WASM targets — it cannot be re-exported via
372// `pub use` because trait impls are not nameable items; callers that
373// want `?` to auto-convert in `#[wasm_bindgen]` fns must add
374// `use grafeo_loro::wasm;` (the impl is in scope via the module).
375#[cfg(feature = "wasm")]
376pub use wasm::error_code;
377#[cfg(all(feature = "wasm", target_family = "wasm"))]
378pub use wasm::{init_panic_hook, js_error};
379
380// Re-export native crates so raw handles are usable immediately (issue #1
381// item 4: Onde receives the `LoroDoc` from `GrafeoLoroApp::doc()` and calls
382// native Loro APIs directly via this re-export).
383#[cfg(feature = "grafeo")]
384pub use grafeo;
385#[cfg(feature = "bridge")]
386pub use loro;