ingraph-codegen 0.1.0

The emission engine behind ingraph-derive: a plain declaration model in, the ingraph framework faces out.
Documentation
//! The AGGREGATE declaration's model — the three parts every aggregation has, read off a grammar
//! and held until the emission arrives.
//!
//! ```ignore
//! #[ingraph::aggregate(
//!   upstream(node = "demux::Track"),
//!   group(codec = "codec"),
//!   measure(count = counting, bytes(fold = summing, from = "size")),
//!   poll(recount, every = "10m", light),
//!   sqlite,
//! )]
//! pub struct CodecCensus {
//!   #[ingraph(group)]   codec: Codec,
//!   #[ingraph(measure)] count: ingraph::Count,
//!   #[ingraph(measure)] bytes: u64,
//! }
//! ```
//!
//! # The grammar NAMES the sentence, and the sentence is `SELECT count(*) FROM t GROUP BY codec`
//!
//! `upstream(…)` is the `FROM`, [`GROUP`] is the `GROUP BY`, and [`MEASURE`] is the aggregate
//! functions. The two words beside them are the ones SQL has no room for and a running system
//! cannot do without: the CLOCK that decides when the answer is recomputed, and the STORE the
//! answer lands in.
//!
//! # It consumes TABLES rather than messages, which is the whole safety argument
//!
//! The framework's delivery correctness is at-least-once plus keyed replace, so duplicates are
//! NORMAL operation — fan-out clones per edge, a rewind re-delivering the unacked tail, a retry
//! re-sending — and no app-visible message id exists to dedup on. An incremental `count += 1`
//! cannot inherit that theorem. So a trigger reads the upstream's ROWS, groups them, folds each
//! group and upserts one row per group: immune to duplicates by construction, self-healing on every
//! tick, and deletion-correct for free, a removed upstream row simply ceasing to be counted.
//!
//! # THE MODEL IS HERE AND THE EMISSION IS BESIDE IT
//!
//! The grammar in `ingraph-derive` builds this model and every rule a declaration can break is
//! refused there by name; the front end then MOUNTS it — the `(group…, measure…)` columns, keyed by
//! the group projection, handed to the ordinary record pass — and hands both halves back as
//! [`Mounted`], which is what expands. Sentences stay the front end's, sentences being the front
//! end's; what belongs here is the model, and beside it in [`emit`] the one thing a record cannot
//! say: what a group's rows fold into.

/// The head block that names the GROUP columns, and the field marker that wears the same word.
///
/// ONE word in two positions, which is [`MEASURE`]'s shape too and the house's own: `searchable` is
/// a full-text declaration's field marker and an entry inside a `vector`'s `knn(…)`, meaning the
/// same thing in both. The block says WHERE a group column is read from, the marker says WHAT THE
/// FIELD IS, and a declaration whose two halves disagree is refused by name rather than resolved by
/// precedence.
pub const GROUP: &str = "group";

/// The head block that names the MEASURE columns, and the field marker that wears the same word.
///
/// The word `fold` was unavailable AS A BLOCK WORD — `semantics(fold(…))` is set-similarity
/// combination, one grammar over — and the collective spelling the design sketch carried (`census`)
/// was settled against the house register by the grammar task: every block word in this family
/// names WHAT AN ENTRY IS (`upstream`, `lineage`, `poll`, `graphql`, `group`), never what the
/// declaration as a whole amounts to, and a census is the whole. So the block takes the entry's own
/// noun, and takes it in the singular, which is `upstream(<field> = "…")`'s rule for a block holding
/// N entries.
///
/// **`fold` returns one grain in as a PROPERTY of an entry** — `measure(<field>(fold = …, from =
/// "…"))` — and that is not the collision the block word dodged. A property word is read inside one
/// entry of one block, where the set-similarity word is an entry of `semantics(…)`; neither reader
/// can see the other's position, and what the word means in both is the same thing said about a
/// different subject.
pub const MEASURE: &str = "measure";

/// A validated aggregate declaration.
///
/// PLAIN, for the reason every model in this crate has plain fields: the derive today and a
/// manifest front end later reach the same emitter.
///
/// # Three fields are TOKENS, and the passthrough is the journal's law rather than laziness
///
/// [`triggers`](Self::triggers), [`stores`](Self::stores) and [`graphql`](Self::graphql) are the
/// words this declaration shares with the RECORD grammar, and they are carried UNINTERPRETED. The
/// aggregate's storage IS a record's — the emission mints the `(group…, measure…)` row shape and
/// hands it to the record pass, which is `#[ingraph::journal]`'s pattern verbatim — so reading a
/// database block, a `table = "…"` or a `poll(…)` entry here would be a SECOND authority for one
/// vocabulary, and two authorities for one vocabulary is how the two of them drift.
///
/// What the front end knows about them is only which WORDS start one, which it needs anyway to
/// refuse a declaration that names no clock. Their teeth arrive with the record pass, at the same
/// expansion that spends them.
#[derive(Debug, Clone)]
pub struct AggregateModel {
  /// The declaration's own name.
  pub aggregate: syn::Ident,
  /// The declaration's visibility, carried for the types the emission mints beside it.
  pub vis: syn::Visibility,
  /// Where `ingraph` lives, as `crate = "…"` named it or `::ingraph` where it named none.
  pub krate: syn::Path,
  /// The same, AS WRITTEN — the literal the record grammar is handed back when the emission mounts
  /// this declaration as a record.
  ///
  /// Both halves are carried for `#[ingraph::journal]`'s stated reason: one authority for the path,
  /// and no round trip through a rendered path that could differ from the one that was written.
  pub krate_spelled: Option<syn::LitStr>,
  /// `upstream(node = "…")` — the ONE node whose rows are folded.
  ///
  /// The edge and NOTHING ELSE, which is where this word differs from the derived indexes' spelling
  /// of it: theirs carries the projection too, because a document is one upstream message and every
  /// field of it is read out of that message. Here only the GROUP columns are read out of a row —
  /// a measure's column is FOLDED into being — so a projection written inside `upstream(…)` would
  /// claim a totality this kind does not have, and is refused pointing at [`GROUP`].
  pub upstream: syn::Path,
  /// The GROUP projection, in the order the declaration wrote its fields.
  ///
  /// Never empty: a declaration with no group is refused before a model is built. The key this kind
  /// mints IS these columns, which is why a groupless aggregate has no shape here — it would be one
  /// keyless row, and the record family stores rows by key.
  pub groups: Vec<Grouped>,
  /// The MEASURE columns, in the order the declaration wrote its fields. Never empty.
  pub measures: Vec<Measured>,
  /// Every `poll(…)` entry as written, comma-joined, ready to be handed to the record grammar.
  ///
  /// Never empty: an aggregate recomputes ON A TRIGGER, so a declaration naming no clock would
  /// never run and its table would answer nothing, forever.
  pub triggers: proc_macro2::TokenStream,
  /// The store words and their two common entries as written, comma-joined — `sqlite`,
  /// `postgres(schema = "…")`, `table = "…"`, `rename = "…"`.
  ///
  /// May be EMPTY, and the emptiness is answered by the record pass rather than here: *a thing that
  /// holds rows names where they land* is the record grammar's sentence, and this grammar knowing
  /// only which words START a store has nothing of its own to add to it.
  pub stores: proc_macro2::TokenStream,
  /// `graphql(…)` as written, or [`None`] where the declaration named no backend.
  pub graphql: Option<proc_macro2::TokenStream>,
}

/// One GROUP column: a field of this declaration, and the upstream field it is read out of.
#[derive(Debug, Clone)]
pub struct Grouped {
  /// The field of this declaration being filled.
  pub field: syn::Ident,
  /// The upstream field it is read out of, carrying the span of the string that named it — so
  /// every claim the emission makes about the upstream lands where the author wrote it.
  pub from: syn::Ident,
  /// The field's declared type.
  pub ty: syn::Type,
  /// How an accessor hands this column back — the three projection words, read by the same walk
  /// every tier with columns reads them with.
  pub projection: crate::model::Projection,
}

/// One MEASURE column: a field of this declaration, what folds a group's rows into it, and the
/// upstream column that fold is handed.
#[derive(Debug, Clone)]
pub struct Measured {
  /// The field of this declaration being filled.
  pub field: syn::Ident,
  /// What folds the group into this column.
  pub by: Measure,
  /// The upstream column whose values the fold is handed, where the entry named one — carrying the
  /// span of the string that named it, so every claim the emission makes about the upstream lands
  /// where the author wrote it.
  ///
  /// # [`None`] is a FOLD OVER THE ROWS, and it is a shape rather than a gap
  ///
  /// A measure's `from` says WHICH values become the slice a fold is handed. Where it is written,
  /// that slice is the named column's values; where it is not, the fold is handed the group's ROWS
  /// — which is exactly what `counting` wants, answering how many there are about items of every
  /// type there is.
  ///
  /// So the two states are two legal declarations and not a filled-or-missing seat, and which of
  /// them a fold may take is [`Measure::owed`]'s answer: the grammar refuses `counting` a column and
  /// refuses `summing` none, and leaves a fold the declaring crate wrote to say for itself. **It is
  /// never DEFAULTED to [`field`](Self::field)** — a measure's own name is this declaration's and
  /// the column is the upstream's, so a grammar that assumed the two agree would fold the wrong
  /// column wherever they did not, silently. [`Grouped::from`] is written out for the same reason
  /// even where the two spellings match.
  pub from: Option<syn::Ident>,
  /// The field's declared type.
  pub ty: syn::Type,
  /// How an accessor hands this column back — [`Grouped::projection`]'s word at the other role.
  pub projection: crate::model::Projection,
}

/// WHAT FOLDS a group's rows into one measure column.
///
/// # The axis is CLOSED on the framework's side and OPEN past it
///
/// A bare word is one of the framework's own shipped folds, and that vocabulary is closed because a
/// word is a promise this crate can write the whole binding — which it can only keep for a fold it
/// SHIPS. A QUOTED PATH is every other fold there is: a type the declaring crate wrote, named the
/// way this grammar names every type it did not mint, with no vocabulary to be admitted to.
///
/// That is the round's own law read at one more position — MINT BARE, REFER QUOTED — and it is why
/// the two forms are one enum rather than two words: an author choosing between a shipped fold and
/// their own is choosing a value for one entry, not writing a different entry.
///
/// The axis is about the fold's ORIGIN and about nothing else, so it is orthogonal to the entry's
/// two spellings: both halves are written the same way at `fold = …` inside the full form, and the
/// sugar `<field> = <fold>` takes both too — where the fold [`owes`](Self::owed) the entry no
/// column, which is the one thing the short spelling has no seat to say.
///
/// # The SHIPPED SPELLINGS are the design's, and the house register disagrees with them
///
/// Every other closed VALUE vocabulary in this grammar is a bare short NOUN — `semantics(fold(sum))`
/// and `fold(max)`, `dedup(hash)` and `dedup(eq)`, the five tracing levels, the four execution
/// worlds — and `counting`/`summing` are the first gerunds in it. The design law one grammar over
/// (*a key names the KIND of the value it takes*; same kind, different reach, one word) would make
/// this position's word for a total `sum`, which is the word `semantics(fold(…))` already spells a
/// fold with.
///
/// The names stand as the closed design wrote them: the design reserved the BLOCK word for the
/// grammar task's census and named these two itself, and the fold seam that mints the citizens is
/// its own task. The measurement is recorded here rather than acted on, and it is one line to act
/// on — [`SHIPPED`](Self::SHIPPED) and [`shipped`](Self::shipped) are the whole of it.
#[derive(Debug, Clone)]
pub enum Measure {
  /// `counting` — how many rows the group holds.
  Counting,
  /// `summing` — the total of the upstream column this measure folds.
  ///
  /// WHICH column is [`Measured::from`], and the entry names it or is refused: the fold reads one
  /// and the framework knows it does, so a `summing` with nowhere to read from is an entry with a
  /// word left out. See [`owed`](Self::owed) for the axis that decides it.
  Summing,
  /// A fold the declaring crate wrote, as `measure(<field> = "path::to::Fold")` or the full form's
  /// `fold = "path::to::Fold"` named it.
  ///
  /// Nothing here resolves it, which is the mechanism rather than a limitation: two macro
  /// expansions cannot see each other, so the path is EMITTED and rustc resolves it at the span of
  /// the string the author wrote.
  Declared(syn::Path),
}

/// WHETHER A FOLD OWES ITS ENTRY A COLUMN — the axis the `from` seat is checked against.
///
/// # It exists because the framework's knowledge of a fold is the SHIPPED half and stops there
///
/// A word this crate ships is a fold this crate wrote, so what it reads is a fact rather than a
/// guess: `counting` answers how many rows there are and looks at none of them, `summing` adds up
/// one column's values. Those two facts are what let the grammar refuse a `counting` that names a
/// column and a `summing` that names none — one mistake apiece, caught at the declaration rather
/// than at a fold handed the wrong slice.
///
/// A QUOTED path is a type this crate never saw. Refusing it either way would be the framework
/// deciding on the declaring crate's behalf, so the third answer says so and neither refusal fires.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Owed {
  /// ONE UPSTREAM COLUMN, which the entry must name — `summing`.
  Column,
  /// NOTHING: the fold is handed the group's ROWS and reads no column of them — `counting`.
  Nothing,
  /// THEIRS to say: a fold the declaring crate wrote, which this crate cannot answer for, so both
  /// spellings are accepted and the emission hands over whatever the entry named.
  Theirs,
}

impl Measure {
  /// The framework's shipped folds, in the order a refusal lists them.
  ///
  /// The near-miss list for a bare word inside [`MEASURE`] is read off this, so the words a
  /// suggestion offers and the words the sentence names cannot drift apart.
  pub const SHIPPED: [&'static str; 2] = ["counting", "summing"];

  /// What this fold OWES its entry, which is what the `from` seat is checked against.
  ///
  /// One arm per citizen and no default arm, so a third shipped fold cannot be added without its
  /// author deciding what it reads — which is the whole of what the entry's grammar needs to know
  /// about it.
  #[must_use]
  pub const fn owed(&self) -> Owed {
    match self {
      Self::Counting => Owed::Nothing,
      Self::Summing => Owed::Column,
      Self::Declared(_) => Owed::Theirs,
    }
  }

  /// The shipped fold a bare word names, or [`None`] where the word is none of them.
  #[must_use]
  pub fn shipped(word: &str) -> Option<Self> {
    match word {
      "counting" => Some(Self::Counting),
      "summing" => Some(Self::Summing),
      _ => None,
    }
  }

  /// The word this fold was written as — a shipped one's own spelling, or the declared path's.
  #[must_use]
  pub fn as_written(&self) -> String {
    match self {
      Self::Counting => Self::SHIPPED[0].to_owned(),
      Self::Summing => Self::SHIPPED[1].to_owned(),
      Self::Declared(path) => quote::ToTokens::to_token_stream(path).to_string(),
    }
  }
}

impl AggregateModel {
  /// The declaration's columns in ROW ORDER — every group column, then every measure.
  ///
  /// The row shape the emission mints is these columns in this order, so the walk is the model's
  /// rather than each reader's: two readers ordering one row shape differently is a table whose
  /// columns depend on who asked. Groups lead because they are the KEY, and a key is a row's first
  /// columns everywhere else in this framework.
  pub fn columns(&self) -> impl Iterator<Item = &syn::Ident> {
    self
      .groups
      .iter()
      .map(|grouped| &grouped.field)
      .chain(self.measures.iter().map(|measured| &measured.field))
  }
}

mod emit;

pub use emit::Mounted;

#[cfg(test)]
mod tests;