Skip to main content

Module aggregate

Module aggregate 

Source
Expand description

The AGGREGATE declaration’s model — the three parts every aggregation has, read off a grammar and held until the emission arrives.

ⓘ
#[ingraph::aggregate(
  upstream(node = "demux::Track"),
  group(codec = "codec"),
  measure(count = counting, bytes(fold = summing, from = "size")),
  poll(recount, every = "10m", light),
  sqlite,
)]
pub struct CodecCensus {
  #[ingraph(group)]   codec: Codec,
  #[ingraph(measure)] count: ingraph::Count,
  #[ingraph(measure)] bytes: u64,
}

§The grammar NAMES the sentence, and the sentence is SELECT count(*) FROM t GROUP BY codec

upstream(…) is the FROM, GROUP is the GROUP BY, and MEASURE is the aggregate functions. The two words beside them are the ones SQL has no room for and a running system cannot do without: the CLOCK that decides when the answer is recomputed, and the STORE the answer lands in.

§It consumes TABLES rather than messages, which is the whole safety argument

The framework’s delivery correctness is at-least-once plus keyed replace, so duplicates are NORMAL operation — fan-out clones per edge, a rewind re-delivering the unacked tail, a retry re-sending — and no app-visible message id exists to dedup on. An incremental count += 1 cannot inherit that theorem. So a trigger reads the upstream’s ROWS, groups them, folds each group and upserts one row per group: immune to duplicates by construction, self-healing on every tick, and deletion-correct for free, a removed upstream row simply ceasing to be counted.

§THE MODEL IS HERE AND THE EMISSION IS BESIDE IT

The grammar in ingraph-derive builds this model and every rule a declaration can break is refused there by name; the front end then MOUNTS it — the (group…, measure…) columns, keyed by the group projection, handed to the ordinary record pass — and hands both halves back as Mounted, which is what expands. Sentences stay the front end’s, sentences being the front end’s; what belongs here is the model, and beside it in [emit] the one thing a record cannot say: what a group’s rows fold into.

Structs§

AggregateModel
A validated aggregate declaration.
Grouped
One GROUP column: a field of this declaration, and the upstream field it is read out of.
Measured
One MEASURE column: a field of this declaration, what folds a group’s rows into it, and the upstream column that fold is handed.
Mounted
A MOUNTED aggregate: the record it IS, and the declaration that folds into it.

Enums§

Measure
WHAT FOLDS a group’s rows into one measure column.
Owed
WHETHER A FOLD OWES ITS ENTRY A COLUMN — the axis the from seat is checked against.

Constants§

GROUP
The head block that names the GROUP columns, and the field marker that wears the same word.
MEASURE
The head block that names the MEASURE columns, and the field marker that wears the same word.