ingraph 0.1.0

A declaration-driven indexing framework: records and nodes are written as Rust types, and the storage rows, the dataflow between them and the GraphQL query surface are emitted from the declaration.
Documentation
//! The storage seams: what a store IS, what one call against it receives, what a search hands back,
//! and — for the families that have arrived — what a blob store, a vector index and a full-text
//! index do.
//!
//! Two of those are family-free and sit at this level in files of their own: [`Connected`], the
//! resources every op takes, and [`Scored`], the hit every search returns. Neither belongs to an
//! engine, and a copy per family would be types that must agree and are free to stop.
//!
//! A seam here is a trait a BACKEND implements, never one this crate implements for anybody. The
//! division that makes it two layers rather than one is the connection: a `*Store` is opened once
//! and IS the manager, and the per-slot handle it mints is stateless and reaches back through
//! [`Connected`] — so nothing reconnects per call and a handle can be cheap enough to make per op.
//!
//! # Three families are here, and the fourth is not
//!
//! Meta stores have the same two-layer shape and are not written, because nothing asks for one.
//! That is [`record`](crate::record)'s rule one layer down.
//!
//! Blob asked first: its address is derivable from the key alone, so it needs no query language, no
//! element vocabulary and no document schema to be useful. Vector asked second, once
//! [`Embedding<E, N>`](crate::Embedding) supplied the element vocabulary it had been waiting on —
//! and the wait was for a vocabulary of 721 lines, which the type that replaced it does not need.
//! Full-text asked third, and what it had been waiting on was nothing of the sort: its record faces
//! ([`FulltextRecord`](crate::FulltextRecord), [`FulltextTable`](crate::FulltextTable)) landed
//! BEFORE the seam did, and what arrived last is a live engine to spell one against.
//!
//! An index SEARCHES, which a blob store does not, and the seam carries that verb without carrying a
//! query language: [`VectorRead::knn`] takes coordinates (and the [`VectorColumn`] they are
//! measured against, a declaration being free to elect several) and [`FulltextRead::search`] takes
//! a query string (and the [`FulltextSubset`] of elected columns it runs over, for the same reason),
//! and both hand back scored keys, while every knob a search is shaped by — `k`, a metric, a
//! probe count, a hit count, an analyzer — is read off the backend's own
//! [`ReadContext`](VectorRead::ReadContext), which is a type the backend picked. The two verbs differ
//! in exactly one thing, and it is what each family's engines already agree about: coordinates are a
//! neutral shape and a query GRAMMAR is not, so the vector side spells its query and the full-text
//! side passes a `&str` through to whichever grammar the engine has. The SCOPE is deliberately not
//! passed that way — a field-scoped query written into the string is a magic string a caller
//! composes and nobody checks — which is why it rides beside the text as a typed argument. What has
//! not arrived on either is the layer ABOVE the verb, where a search is composed, ranked and fused
//! across families. Nothing here names it, and the half a backend implements is this one.
//!
//! **The vector side carries a read verb the full-text side does not**: [`VectorRead::get`] hands
//! an entity's OWN vectors back at an address, which is what makes *similar to THIS entity*
//! answerable without the framework re-projecting a record it has no store hop to fetch. The
//! asymmetry is a SCHEDULE — a full-text index answering *what did you store for this key* is the
//! `stored` marker's own question, whose grammar has not arrived — rather than a claim that one
//! family reads by key and the other cannot.
//!
//! # The verbs have FACES, and a bound is what picks one
//!
//! A single engine trait carrying every verb its family has is a bound that asks for all of them.
//! The EIGHT FACES are that seam split along the two axes its own members already differ on —
//! direction (the read verbs take a [`ReadContext`](VectorRead::ReadContext), the write verb a
//! [`WriteContext`](VectorWrite::WriteContext)) and mode ([`VectorRead`] against
//! [`AsyncVectorRead`]). A retrieve binds a read face, a drainer binds a write one, and an engine
//! that only reads becomes spellable.
//!
//! # The COLUMN LANGUAGE, and the bases it left behind
//!
//! A key crosses into an engine's address vocabulary one TYPED COLUMN at a time
//! ([`ColumnWriter`] / [`ColumnReader`]), so a key's row ([`Field`]) names no engine and every engine
//! reads every row. [`VectorEngine`] and [`FulltextEngine`] are what that made of a base: the
//! vocabulary, plus the three phases ([`buffer`](VectorEngine::buffer),
//! [`finalize`](VectorEngine::finalize), [`open`](VectorEngine::open)) that build an address and
//! read one back — no verb, no lease, no GAT. BOTH families' per-engine row traits are deleted, and
//! the thirty-seven rows one of them shipped are one vocabulary now.
//!
//! [`round_trip_vector`] and [`round_trip_fulltext`] are the FACTORY PART that comes with it: the
//! round-trip theorem as a function a backend author runs over their own keys, one entry per family
//! and one body. It also carries the half no engine can check for itself — that a row asks back for
//! the KIND of column it landed — which is invisible wherever
//! [`fixed`](ColumnWriter::fixed) is still the defaulted one and wrong on the first engine that
//! overrides it.
//!
//! # What a backend chooses, and why none of it is narrowed here
//!
//! SEVEN surfaces are the backend's: [`AsyncBlobEngine::Payload`], [`AsyncLocate::Ref`], [`AsyncLocate::Location`],
//! [`AsyncBlobStore::Options`], [`AsyncVectorStore::Options`], [`AsyncFulltextStore::Options`] and every context
//! type. Each is an
//! associated type with no bound, and each has a second implementation already in view that would
//! break under the obvious narrowing — a remote store hands back a refcounted body where the
//! filesystem hands back a `Vec<u8>`, and answers to client options where the filesystem answers to
//! a root directory. Narrowing them is refused in advance rather than argued again per backend.
//!
//! [`AsyncLocate::Location`] is the one of the seven a CLIENT sees, and it is bare for the reason the
//! other six are: what a store knows about where its bytes live — a size, an etag, a generation, a
//! signed expiry — is that store's fact and not a shape this crate can enumerate. [`BlobLocation`]
//! is the convenience a backend with nothing further to say elects, and it is A citizen of that seat
//! rather than its definition.

mod blob;
pub(crate) mod column;
mod conformance;
mod connected;
mod faces;
mod fanout;
mod field;
mod fs;
mod fulltext;
mod handle;
mod knobs;
mod location;
mod mount;
mod scored;
mod slot;
#[cfg(feature = "tantivy")]
mod tantivy;
mod vector;

pub use blob::{AsyncBlobEngine, AsyncBlobStore, AsyncLocate, BlobEngine, BlobStore, Locate};
pub use column::{ColumnError, ColumnReader, ColumnWriter, NoColumns};
pub use conformance::{ColumnDoor, ConformanceError, round_trip_fulltext, round_trip_vector};
pub use connected::Connected;
pub use faces::{
  AsyncFulltextRead, AsyncFulltextWrite, AsyncVectorRead, AsyncVectorWrite, FulltextRead,
  FulltextWrite, VectorRead, VectorWrite,
};
pub use fanout::Refused;
pub use field::Field;
pub use fs::{
  KeySegments, LocalFsBlob, LocalFsBlobError, LocalFsBlobRecord, LocalFsStore, MAX_PATH_LEN,
  MAX_SEGMENT_LEN, PathTooLong, Segment, SegmentTooLong, Segments, blob_address,
};
pub use fulltext::{
  AsyncFulltextStore, FulltextColumn, FulltextEngine, FulltextLane, FulltextRoad, FulltextStore,
  FulltextSubset, IndexRefusal,
};
pub use handle::SlotHandle;
pub use knobs::Noop;
pub use location::BlobLocation;
pub use mount::{FulltextMount, VectorMount};
pub use scored::{Hits, Scored};
pub use slot::{Bound, Slot};
#[cfg(feature = "tantivy")]
pub use tantivy::{
  KeyDecodeError, LaneAnalyzers, SlotMismatch, SlotWriter, TantivyAddress, TantivyColumns,
  TantivyDoc, TantivyFulltext, TantivyFulltextError, TantivyStore, TantivyTerms, UnknownAnalyzer,
};
// The slot maker takes the LAUNCH's word on top of the backend's, its own type parameter being the
// shareable-resource layer's `AsyncResource`. See the type for why the rest of the backend needs
// neither.
#[cfg(all(feature = "runtime", feature = "tantivy"))]
pub use tantivy::TantivySlotMaker;
pub use vector::{AsyncVectorStore, Elected, VectorColumn, VectorEngine, VectorStore};

/// THE KIND CHECK a full-text leg's `resource = "…"` is verified by, as a function an expansion
/// takes the address of.
///
/// `const _: fn() = holds_documents::<crate::Ft>;` beside the leg, which is the shape every audit in
/// this crate takes — see [`stores_rows`](crate::stores_rows), where the idiom is written down.
///
/// # Why a named function where a projection would already refuse
///
/// A pointed leg that mints a query package is verified for free: the package's `floor` row is
/// `<Ft as FulltextEngine>::Score`, so a resource of the wrong kind is `E0277` inside it. But the
/// package is emitted only where a declaration ASKS for one — an unfanned, unlabelled or purely
/// written leg publishes no seat at all — and a check that runs only where something else happened
/// to need it is not a law. This runs for every leg that points.
///
/// It is also where the SPAN is bought. The projections sit inside package rows the author never
/// wrote, so `E0277` lands on the attribute; this is emitted `quote_spanned!` at the path itself, so
/// the caret sits on the word — which is the whole of what *named → kind-verified, at the word*
/// means.
///
/// The bound is the BASE and not a face: verifying a kind must not drag in a verb obligation, which
/// is exactly what the base being verb-free is for. A store that can hold documents passes here
/// whether or not it also writes them.
pub fn holds_documents<E>()
where
  E: FulltextEngine,
{
}

/// THE KIND CHECK for a vector leg — [`holds_documents`]'s twin, one family over and unchanged in
/// every argument.
pub fn holds_coordinates<E>()
where
  E: VectorEngine,
{
}

/// The SLOT NAME's transport type, re-exported because the per-slot factories already name it.
///
/// [`FulltextStore::fulltext`], [`VectorStore::vector`] and both [`mount`](FulltextMount::corpus)
/// verbs take `impl Into<Utf8Bytes>`, so neither a store impl nor a mount impl can be written
/// without spelling it — and neither a MACRO-WRITTEN one nor a THIRD-PARTY one can spell it at all
/// unless it is reachable through this crate: a declaring crate has no `smol_bytes` dependency of
/// its own, and every framework name an expansion writes goes through `__private`. It is at the
/// crate ROOT beside the seams for the mount's sake — the party that writes one is an application,
/// which reaches nothing through `__private`. A re-export rather than a newtype, for the standing
/// reason: the type is already in the seam's public signatures, and wrapping it here would be a
/// second currency for one fact.
pub use smol_bytes::Utf8Bytes;