ingraph 0.1.0

A declaration-driven indexing framework: records and nodes are written as Rust types, and the storage rows, the dataflow between them and the GraphQL query surface are emitted from the declaration.
Documentation
//! RETRIEVE — what a full-text or vector index ANSWERS, above the store seam: two tiers, one
//! pipeline, and no trait over either.
//!
//! ```text
//! Transcript::retrieve_keys(engine, scope, &q, 20).await?
//!     Vec<Ranked<SegmentKey, Score>>          the engine alone
//!
//! Transcript::retrieve(engine, scope, &mut store, &q, filter, 20).await?
//!     Vec<Ranked<Segment, Score>>             the WHOLE upstream entities
//! ```
//!
//! A VECTOR index spells that pair PER ELECTION — `retrieve_keys_by_<field>` /
//! `retrieve_by_<field>` — because it may elect several embeddings, and each is a column of its one
//! slot. A FULL-TEXT index spells the same pair per election, for the same reason and under the same
//! names: it elects too, with `#[ingraph(searchable)]`, and
//! [`FulltextRead::search`](crate::FulltextRead::search) takes a
//! [`FulltextSubset`](crate::FulltextSubset) beside the query, which is where a per-field door puts
//! its answer.
//!
//! What the full-text kind has BESIDE that family, and the vector kind does not, is the UNSUFFIXED
//! pair — `retrieve_keys` / `retrieve`, asking `All`. It is not the N = 1 case's alias: `All` is
//! whatever the SLOT holds, which is a different question from *this field*, so a declaration
//! electing one field emits both.
//!
//! Beside each election's pair sit TWO MORE doors, both taking a KEY where the pair takes
//! coordinates:
//!
//! ```text
//! Voiceprint::get_embedding(slot, own, &key).await?
//!     Embedding<Element, N>                   the entity's OWN vector, CHECKED — a read
//!
//! Voiceprint::retrieve_similar_by_embedding(slot, own, scope, &key, floor, 20).await?
//!     Vec<Ranked<SegmentKey, Score>>          the entity's neighbours, WITHOUT the entity
//! ```
//!
//! `get_<field>` is the by-key read raised to door grade, and it is the SINGLE AUTHORITY on what an
//! engine stored for a key: the seam answers a SET, an election declares ONE, so none, several and a
//! width the declaration does not state are all named there. One engine call, one read scope, no
//! dedup bound.
//!
//! `retrieve_similar_by_<field>` is sugar and not a second query path — *fetch own set → query with
//! it → exclude self*, each step a call to something already here, the first of them being the door
//! above. The exclusion is the DEDUP STAGE with the example already seen
//! ([`ranked_keys_excluding`]), which is what keeps the example's raw-stream ordinal a HOLE and
//! makes `limit` count neighbours. What both can refuse that the pair cannot is the KEY's own stored
//! vector, and that vocabulary is [`OwnReadError`] — shared, because a judgment made in one door and
//! reported by two has to be one type or the delegation would need a translation.
//!
//! # An election LABELLED as a set grows a third door
//!
//! A field labelled `list(embedding)` elects a per-entity SET of embeddings, and a set is asked with
//! a set: `retrieve_keys_by_<field>_with` takes `m` query vectors and a [`FoldSemantics`], and
//! answers in [`Folded`] rather than in [`Ranked`] — the fold authors that answer's order, so no
//! ordinal is minted ACROSS the `m` engine streams and each stream's own is carried attributed to
//! it, in a [`StreamBest`] beside the score that stream measured with. The pair above is untouched
//! and keeps answering the single-vector question, which the `m = 1` coincidence makes the same
//! question.
//!
//! ```text
//! Frames::retrieve_keys_by_frames(engine, scope, &q, 20).await?
//!     Vec<Ranked<KeyframeKey, Score>>          one query vector, the engine's own ordinal
//!
//! Frames::retrieve_keys_by_frames_with(engine, scope, &qs, Sum::new(), 20, None, None).await?
//!     Vec<Folded<KeyframeKey, Score>>          m query vectors, folded, k counting ENTITIES
//! ```
//!
//! The fold is [`set_scored`] and the collapse beneath it is [`owner_best`]; what they spend is a
//! [`FoldSemantics`] — a TYPE rather than a word, of which [`Sum`], [`Max`] and [`Weighted`] are the
//! three this framework ships.
//!
//! # Two tiers, and the second is a COMPOSITION rather than a second query kind
//!
//! **Tier A** is the engine alone: hits in, ranked keys out, no store hop and no filter — a filter
//! needs rows and a key has none.
//!
//! **Tier B** is tier A plus the generic gather the record lanes already run, with the keys as a
//! predicate: ONE statement carrying `WHERE <row-value keys> AND <the client's filter>`, rows
//! crossing to entities through the existing read legs, and the order restored FROM THE KEY LIST.
//! Nothing about it is a new query machine — it is the relation/derived resolver's own move, hosted
//! one layer below the wire so a Rust-only consumer gets the same composite.
//!
//! # The pipeline, in its ruled order
//!
//! ```text
//! engine  →  keyed  →  floor  →  dedup  →  rank  →  tier A  →  gather ∧ filter  →  tier B
//! ```
//!
//! Every stage is stated where it lives: the address read at [`keyed`] and [`vector_keyed`], the
//! floor and the walk at
//! [`ranked_keys`], the mechanism at [`Seen`], the numbers at [`Ranked`], the fill loop at
//! [`filled`]. The address read is FIRST because A′ made it so: the engine seams name no record, so an
//! engine answers with its OWN addresses and the typed read is the emission's — and everything below
//! it is written against keys. What the module note adds is
//! why the ORDER is not a taste — each stage is the precondition of the next. The floor terminates
//! the walk because the stream is descending; dedup runs on the engine's own key list because that
//! is where a duplicate is still cheap; rank is minted over the raw stream because a number counted
//! after a prune is a number that cannot be continued from.
//!
//! # There is no `Retrieve` trait, and that is r1's ruling rather than an omission
//!
//! The face is EMITTED, per node, as inherent methods. A node's filter type, its upstream entity
//! and its dedup mechanism are all its own, so a unifying trait would have to be generic over
//! everything that differs and would name nothing that is shared. What IS shared is this module:
//! the orchestration is written once, framework-internal, generic over the engine call and over the
//! dedup mechanism, and the emitted methods delegate to it. A trait minted OVER those methods, if
//! the fused cross-index query ever needs one, is a pure widening.
//!
//! That widening has since landed, and the caller was the arm rather than the fused query:
//! [`Serves`](crate::Serves) / [`AsyncServes`](crate::AsyncServes) is the read path's capability
//! seam in both modes, and [`SlotLane`](crate::SlotLane) is the framework's own implementation of
//! its AWAITED face — honestly so, its body being the engine's own async `search`/`knn` — the same branded
//! `search`/`knn` the methods below call, wearing a trait so a third party can wrap it. NOTHING
//! here changed for it: the ruling above still holds of everything ABOVE the engine call, which is
//! why the shipped lane stops at that call and leaves the address read, the floor, the dedup and
//! the rank where the declaration is concrete.
//!
//! # ONE query, N indexes — the fold above all of this
//!
//! Everything above answers ONE index. The seventh query kind asks all of them at once and fuses
//! what comes back, and its face-free half is [`fuse`] — the strategy, the fused shape, the join's
//! mechanism and the two per-member limit specs. It is a fold OVER this pipeline rather than a stage
//! inside it: each member runs the walk above, its survivors are projected to their common root, and
//! reciprocal rank fusion reads the ordinals [`Ranked::rank`] carries.
//!
//! # ONE QUESTION, N QUESTIONS — the step at the HEAD of the road
//!
//! Everything above starts from a question somebody asked. The [`Expansion`] seat is what may come
//! first: it holds an optional [`AsyncQueryExpander`], and what it answers is the ROAD — the
//! caller's own question pinned at index `0`, then whatever the mounted expander added.
//!
//! It is a fold over the pipeline exactly as the fusion above is, and it ENTERS that fold rather
//! than standing beside it: each entry of the road walks the tower and the member it always walked,
//! so N entries are N ranked streams and [`fuse`](fuse()) takes them as the members they already
//! are. Nothing about the machinery above knows it happened.
//!
//! **An absent seat is the IDENTITY road** — one entry, the caller's own value — so a deployment
//! that mounts nothing walks exactly the one walk it walked before the seat existed. That is the
//! property the whole step is measured against, and `tests/tower_expand` asserts it against the bare
//! walk itself.
//!
//! # ONE QUESTION, THE SAME QUESTION — the stage before that one
//!
//! [`AnnotatedQuestion`] is the seat a stage EARLIER than the widening writes into. A question may
//! be handed to a detector and come back AS ITSELF, carrying one more fact; the widening's targets
//! and the crossing's lanes then read that fact off the question they were already going to be
//! handed. It is the one stage whose ends are the same type, and that is what lets its datum travel
//! FORWARD to stages a step's answer could never reach.
//!
//! **The caller is supreme over it.** A question that already carries the annotation is not
//! re-annotated and the stage's operator is not consulted — see [`annotate`] for why that reading
//! rather than the one that spends a model forward pass on a value it will discard.
//!
//! The TREE that fold runs over is mounted: [`Fusable`] is the seam a member is reached through,
//! [`Route`] is the edge a parent holds one route by, and [`FusionOptions`] is the flavour a client
//! picks per tier — with the packages themselves emitted, at slot and tier grain, by the declarations
//! this crate has never seen. The DOORS that spend all of that are still emissions nobody writes yet,
//! which is the one half of the seventh query kind that has not landed.
//!
//! # THE ONE-SENTENCE LAW ABOUT NUMBERS: which of them may cross the wire
//!
//! **Protective numbers never cross the wire; preference numbers do.**
//!
//! A `depth`, a `floor`, a fold's flavour and an engine's own knobs are PREFERENCES — a requester's
//! statement about the answer it wants — so they ride the request, and a client rewriting one is a
//! client asking a different question. A CEILING is the one number of a different nature: it is an
//! upper bound on what any single request may cost this deployment, and *a bound the requester can
//! rewrite binds nothing*. So it alone leaves the request surface. Its home is the declaration —
//! deployment configuration being a future additive source — and it CAPS rather than replaces, which
//! is [`route_depth`]'s `min(depth.unwrap_or(limit), ceiling)` in one line.
//!
//! The law is what the request packages are shaped by: every row a declaration mints on one is a
//! preference, and there is no row for a protection to have been written in.
//!
//! # THE THREE-TIER LAW: where each of those numbers is written, and which writing wins
//!
//! The law above says which numbers may cross. This one says how many places each is written in:
//!
//! 1. **a shipped constant** — the framework's own answer, named as a `DEFAULT_*` const;
//! 2. **an assembly override** — what a deployment building the thing names once, at launch;
//! 3. **a query-time parameter** — what one request names, for itself.
//!
//! **A PREFERENCE rides all three and the outermost writing wins outright.** A PROTECTION rides tiers
//! one and two whole, and joins tier three in MIN-TIGHTEN FORM ONLY — the request may narrow the
//! deployment's number and can never widen it, which is [`route_depth`]'s CAPS-not-replaces read in
//! the other direction. Nothing else about the law changes between the two families.
//!
//! Two properties make the tiers composable rather than three chances to disagree:
//!
//! - **an absent tier DEFERS and never defaults.** A silent request composed with a deployment's
//!   value leaves that value standing; the shipped constant is reached at the END of the chain, where
//!   there is nothing left to defer to. Resolving early would make *said nothing* and *asked for the
//!   shipped number* the same request, and the assembly tier would then lose to every caller who
//!   never mentioned the knob.
//! - **the composition is FIELD BY FIELD**, so a request naming one number of a seat has not
//!   silently restated the others.
//!
//! The compositions are one function per family, spent wherever the family is:
//! [`ArmWeights::over`] and [`FloorSpec::over`] for preferences, [`tightened_budget`] for the
//! protective one.
//!
//! ## ⟨BOUNDARY⟩ TIER THREE HAS A ROAD FOR TWO FAMILIES AND NONE FOR THE THIRD
//!
//! Tiers one and two are landed for every knob this pipeline has. Tier TWO has its deployment seat —
//! [`TowerDoor`](crate::TowerDoor) holds the `<Tower>StepOptions` a launch names once, and the emitted
//! resolver hands it to [`AssembledTower::run`] — and tier THREE now has its carriage:
//! [`DeclaredTower::SearchOptions`] is the word-keyed package a REQUEST names, published as the
//! `options` argument of a dressed recipe's fields and handed to the same run. What each family gets
//! out of it still differs, so they are stated one at a time rather than under a heading that would
//! flatten them:
//!
//! | family | tier 1 | tier 2 | tier-3 CARRIAGE | where the number BITES |
//! |---|---|---|---|---|
//! | arm weight ([`ArmWeights`]) | [`DEFAULT_ARM_WEIGHT`](ArmWeights::DEFAULT_ARM_WEIGHT) | the widening step's row of `<Tower>StepOptions` | the `armWeight` row of `<Tower>SearchOptions` — REAL | the fan walk's ② seat — REAL |
//! | lane emphasis ([`ScriptWeightedFan`](crate::ScriptWeightedFan)) | [`DEFAULT_EMPHASIS`](crate::ScriptWeightedFan::DEFAULT_EMPHASIS) | `with_emphasis` at the assembly | the `laneEmphasis` row, for the SHIPPED word only — REAL | the crossing's own probe weights — REAL |
//! | floors ([`FloorSpec`]) | the declaration's own `floor = "…"` | **none** | **none** | **nowhere** |
//! | budget ([`tightened_budget`]) | none — DEFAULT-less by ruling | [`TowerDoor::with_budget`](crate::TowerDoor::with_budget), beside the mint | the `budget` row — REAL | [`walk_within`], round the whole run — REAL |
//!
//! **The arm weight has all three now.** `Expansion<Q>`'s `Serve::Options` IS [`ArmWeights`], so the
//! ② seat is this framework's own type and a deployment fills it through the tower's step-keyed
//! object; the request's own number rides the `armWeight` row of the word-keyed one, and the emitted
//! run lays the second over the first with [`ArmWeights::over`] before the walk reads a coefficient.
//! What the 2026-08-27 census measured as missing — a parameter on an OPEN seam end users implement —
//! is exactly what was added, by ruling rather than by drift.
//!
//! **The lane emphasis reaches its third tier without its trait moving**, which is the same law
//! answered by a different road. The number lives on a third party's type ([`LaneRouter`] is a
//! surface end users extend), so widening the trait would have taxed every third-party router for a
//! knob only the shipped ones have. What happens instead is that a tower's steps are MINTED PER READ:
//! the emitted run retunes THIS REQUEST's own router with `with_emphasis` before the crossing, and
//! the row is minted only where the `route(…)` word named the SHIPPED weighted fan — a user router's
//! recipe grows no field, so there is no knob that could be published and never spent.
//!
//! **The floors have neither a seat nor a bite, and the recipe's own word records a FACT rather than
//! a value.** [`FloorSpec`] is constructed nowhere outside its own tests, and
//! [`Member::is_floored`] says in as many words that a recipe carries *a floor const was named* and
//! never the const. Nor is there a seam below it to spend one at: a fanned
//! tower's tail answers a [`RankedStream`], which is a stream somebody has ALREADY floored, dedup'd
//! and ranked — the shipped serving lane answers raw `Hits`, and neither of the two asks it is asked
//! with carries a floor at all. One grain over, the CURRENCY a floor is stated in is not projectable
//! at the recipe grain either: [`Fusable`] publishes `Options` and `Question`, [`AsyncFusedRead`]
//! publishes `Root`, `Evidence`, `Error` and `Ctx`, and a member's own score currency lives inside the
//! package a recipe holds BY PROJECTION and cannot read a row out of. So the type's watch stands, and
//! it stands on a measurement rather than on a queue position.
//!
//! **The budget has a seat and a bite, and only its carriage is missing** — which is the census's own
//! finding acted on rather than repeated. Its tier two is NOT a row of `<Tower>StepOptions`, and that is
//! the ruling rather than a workaround: that object carries one row per STEP and a budget bounds the
//! WHOLE WALK, so a row of it would have had to pick a step to belong to. It is named beside the mint
//! instead — [`WalkBudget`] on the tower's own door — and it bites at [`walk_within`], which wraps the
//! one uniform run and therefore covers every stage inside it, fan cells included.
//!
//! Two things had to be answered for that to be honest rather than decorative. The CLOCK: nothing
//! runtime-neutral times an await and this crate names no runtime, so the timer is a seam a deployment
//! fills — [`WalkClock`], one line over whatever the application already runs on — and it arrives
//! PAIRED with the number so that a budget nothing measures cannot be spelled. The BLOCKING WALK: a
//! tower whose [`ALL_SYNC`](DeclaredTower::ALL_SYNC) is `true` has no suspension point, so a race
//! against it could never be won; the seat reads that `const` and refuses at compile time rather than
//! wrapping a walk in a wall that looks like one and is not.
//!
//! Tier THREE is landed too, and [`SearchBudget`] is the citizen it floors on — the universal minimum
//! every tower's request surface carries, because it is the FRAMEWORK's own knob rather than any
//! step's. It is the `budget` row of every minted `<Tower>SearchOptions`, whatever words the
//! declaration wrote, and [`AssembledTower::asked_budget`] is the projection that reads it back out of
//! an otherwise opaque package. [`walk_within`] already took the request side as a parameter, so what
//! moved when the face landed was one argument and not one line of the tightening law.
//!
//! **And a `()` seat still cannot cross**, which is what keeps the tower's STEP-KEYED options object
//! off any request surface however the families above are answered — and is the whole reason there
//! are two objects rather than one. Whether a step's `Serve::Options` is `()` is a fact the COMPILER
//! holds and the emitting macro does not, and `()` has no wire face, so a per-step row cannot be
//! published by a macro that cannot know which rows are empty. The door that sidesteps it publishes
//! FRAMEWORK-OWNED rows only, keyed by the words the macro that mints them can read — which is what
//! `<Tower>SearchOptions` is, and why it is minted at the TOWER grammar rather than at the recipe's.
//!
//! ## ⟨BOUNDARY⟩ Two of the KEEPER's three knobs have no tier three, and the reason is the GRAIN
//!
//! [`SnapshotKeeper`] brings three protective numbers, and they split rather than agreeing:
//!
//! | knob | tier 1 | tier 2 | tier 3 |
//! |---|---|---|---|
//! | capacity | [`DEFAULT_SNAPSHOT_CAPACITY`] | [`SnapshotKeeper::with_capacity`] | **none** |
//! | TTL | [`DEFAULT_SNAPSHOT_TTL`] | [`SnapshotKeeper::with_ttl`] | **none** |
//! | depth | [`DEFAULT_SNAPSHOT_DEPTH`] | [`SnapshotKeeper::with_depth`] | the request's `depth`, min-tighten |
//!
//! This is NOT the `()`-seat blocker above and it is not a road that has yet to be built. Capacity
//! and TTL are properties of ONE SHARED STORE: every client's snapshots sit in it, so a number one
//! request named would be a number it named for its neighbours — and min-tighten does not rescue
//! them either, since tightening a shared store means evicting somebody else's answer sooner. A
//! wire parameter whose only effect is on a third party is not a request's statement about its own
//! answer, which is what the numbers law's *preference* means and what tier three exists to carry.
//!
//! Depth IS one question's own — *how deep an answer do I want kept* — so it composes through
//! [`route_depth`] like every other wall here, and it is this pipeline's first PROTECTIVE number to
//! land all three tiers: the keeper's ceiling is the wall, the request's `depth` is the ask, and a
//! request past the ceiling is CUT rather than refused.
//!
//! # THE READ IS SINGLE-SHOT, AND PAGING IS THE KEEPER'S — the reason survived the widening
//!
//! There is still no cursor slot on the retrieve path and there is still no resumption contract
//! over the backends, and the reason is untouched: resumption is a per-backend capability — Lucene
//! and Elasticsearch have watermarks, tantivy has an offset rebuild, a raw ANN library has nothing
//! at all — and a uniform contract over that spread would be a promise the framework cannot keep.
//! `limit` is a BUDGET rather than a promise, and a member is asked ONCE.
//!
//! What arrived is the widening this paragraph reserved, and it arrived on the other side of the
//! fold: the framework never asks a backend to resume, it HOLDS THE ANSWER IT ALREADY HAS. See
//! [`SnapshotKeeper`] — a bounded LRU/TTL store of fused rankings, keyed by an opaque
//! [`PageToken`], from which a road cuts one [`Stretch`] at a time. The promise is
//! exact-within-snapshot and session-ephemeral, and every way of losing a snapshot ends in a named
//! [`PageTokenRefusal`] rather than an empty answer.
//!
//! Two things follow, and both are load-bearing:
//!
//! - **A backend that can do nothing pages exactly as well as one with watermarks**, because none
//!   of them is asked. That is why the promise is keepable where a resumption contract was not.
//! - **The re-ask road is still there and is still the honest one for a caller who wants CURRENT
//!   rows.** [`Ranked::rank`] — re-ask with a wider budget and drop everything at or below the last
//!   rank seen — walks a live ranking; the keeper walks a frozen one. A snapshot is exact and
//!   stale, a re-ask is current and may shift under the reader, and neither is the other's
//!   replacement.
//!
//! The g4 `Resumable` capability seam stays a recorded door: a backend that CAN resume would let
//! the keeper hold less, and nothing here forecloses it.

mod address;
mod annotate;
mod assembly;
mod bests;
mod budget;
mod dedup;
mod expand;
mod folded;
mod fuse;
mod gather;
mod held;
mod keep;
mod keys;
mod query;
mod ranked;
mod search;
mod semantics;
mod set;
mod similar;
#[cfg(feature = "sqlx")]
mod sql;
mod tower;

pub use address::{keyed, vector_addressed, vector_keyed};
pub use annotate::AnnotatedQuestion;
pub use assembly::{AssembledTower, SeatsUnfilled, UnfilledSeat};
pub use bests::{Bests, EqBests, HashBests};
pub use budget::{BudgetSpent, SearchBudget, WalkBudget, WalkClock, tightened_budget, walk_within};
pub use dedup::{EqSeen, HashSeen, Seen};
pub use expand::{
  ArmWeights, AsyncQueryExpander, ExpandOrigin, Expanded, Expansion, QueryExpander, TextualQuestion,
};
pub use folded::{Folded, StreamBest};
pub use fuse::{
  AsyncFusedRead, EqJoin, FanStream, FloorSpec, Fusable, Fused, FusedScore, Fusion, FusionOptions,
  Grain, HashJoin, Join, RankedStream, Rooted, Route, RrfOptions, Sighting, asked, fuse,
  fuse_fanned, fuse_weighted, participates, route_depth,
};
pub use gather::{AsyncGather, RetrieveError, filled, folded_filled};
pub use held::Held;
#[cfg(feature = "async-graphql")]
pub use keep::installed_keeper;
pub use keep::{
  DEFAULT_SNAPSHOT_CAPACITY, DEFAULT_SNAPSHOT_DEPTH, DEFAULT_SNAPSHOT_TTL, PageToken,
  PageTokenRefusal, SnapshotKeeper, Stretch,
};
pub use keys::{ranked_keys, ranked_keys_excluding};
pub use query::{Deferred, FulltextQuery, VectorQuery};
pub use ranked::Ranked;
pub use search::{
  Binding, Bridge, DeclaredSearch, Member, Seat, SharedQuestion, TrySharedQuestion,
};
pub use semantics::{FoldSemantics, FoldVisitor, Max, Sum, Weighted};
pub use set::{FoldedQuery, Sightings, owner_best, owner_best_excluding, set_scored};
pub use similar::OwnReadError;
#[cfg(feature = "sqlx")]
pub use sql::{SqlGather, SqlPooledGather};
pub use tower::{DeclaredTower, SearchTower, Step};

#[cfg(test)]
mod tests;