ingraph 0.1.0

A declaration-driven indexing framework: records and nodes are written as Rust types, and the storage rows, the dataflow between them and the GraphQL query surface are emitted from the declaration.
Documentation

ingraph

A Rust framework for building retrieval systems — the kind that ingests media or documents, derives things from them (transcripts, embeddings, thumbnails, full-text documents), stores each derived thing in whatever engine suits it, and answers questions over the whole assembly.

Its one claim, stated before anything else:

cargo check green means the retrieval assembly is connected.

Not compiles. Connected — every hop from the bytes that arrive to the answers that leave is a trait bound the compiler discharged. A vector index pointing at a record that does not exist, a question pipeline whose second step cannot eat the first step's output, an embedding of the wrong width — each is a compile error at the word the author wrote, not a runtime surprise at 3am.

0.1 is a first cut

Everything here may change or disappear in a later 0.x. No API is settled, no migration is promised, and there is no deprecation cycle to rely on.

The surface is not a design that was drawn and then filled in. It is grown demand-driven: an item enters when something live asks for it — the codegen emits a path that has to resolve, a test names a type it cannot compile without, a downstream breaks for the want of it — and nothing is added because it will probably be needed later. That rule is what 0.1 was grown by, and it is what the next version will be grown by too.

Installation

[dependencies]
ingraph = "0.1"

One example

use uuid::Uuid;

#[ingraph::record(sqlite)]
pub struct Media {
  #[ingraph(id)]
  id: Uuid,
  title: String,
  duration_ms: u64,
}

#[ingraph::record(sqlite, lineage(fields(media_id)))]
pub struct Keyframe {
  #[ingraph(id)]
  id: Uuid,
  #[ingraph(copy)]
  media_id: Uuid,
  at_ms: u64,
  caption: String,
}

Media is a root record: no lineage, so its key is its id alone. Keyframe declares its lineage — the path from the root, root-first — so key = lineage ⊕ id spells KeyframeKey { media_id, id } with nothing further said about it.

That expands into MediaKey, KeyframeKey, KeyframeLineage, an accessor family on each, From<Record> and From<&Record> conversions, impl Transformer and impl Identifiable on the record and on the key both, the per-database row and comparison lowering, the order-by vocabulary, the keyset cursor, and the record's query unit.

Features

The crate is cut into tiers, and a build takes the rung it needs:

feature what it adds
derive (default) the words a declaration is written in — implies runtime and flags
runtime (default) the drive layer: nodes, channels, schedules, resources — implies engine
engine the retrieve layer: gather, rank, fold, fusion, search towers
(none) the vocabulary core alone — the value types, the filters, the comparison crossing

On top of a tier sit the opt-in words, all default-off: the vocabularies a declaration may name a field type from (smol_str, jiff, time, half, rust_decimal, regex, mediatime, mediaframe), the backends a record is stored in (sqlite, postgres, mysql, mongo, mongodb, tantivy), the wire face (async-graphql, dataloader), and the observation and executor words (tracing, rayon, tokio-signal).

Platform support

Linux and macOS. Both are built, linted and tested in CI.

Windows is untested rather than broken. The library's own sources contain nothing Unix-specific, so it may well compile there, but the repository cannot make that a claim it verifies: the intake acceptance is driven by the tributaries filesystem watcher, whose OS backends are FSEvents and inotify/fanotify only, so cargo test has no Windows configuration to run. A Windows lane would assert support that no test covers.

Related crates

  • ingraph-derive — the macros, re-exported at this crate's root behind the default derive feature. Name ingraph alone; this is here for the rare consumer that reaches the attributes directly.
  • ingraph-codegen — the emission engine behind the macros. An implementation detail of ingraph-derive, published because it is a dependency of one.

License

ingraph is under the terms of both the MIT license and the Apache License (Version 2.0).

See LICENSE-APACHE and LICENSE-MIT for details.