weirdnet-pipelines
Pipelines as graphs (DAGs) of nodes whose meaning belongs to the application.
Kind: the application's node type. It reads and prints a node spec, says whether the node is a source, a step or a sink, checks tees, and names what a sink lands in. The crate gives no source, step or sink a meaning of its own; their names are the application's strings.Graph<K>: nodes (ID=KIND, with+TEEside saves) and edges (A>B[:NAME], chainsA>B>C), serialized as{nodes: [{id, kind…, tees}], edges: [{from, to, as}]}.Graph::order: every node after its upstreams; a cycle is an error naming its nodes.Graph::check: duplicate ids, edges to missing nodes, self-loops, cycles, no source or sink, sources with inputs, steps or sinks with none, leaves that are not sinks, sinks with something after them, and (with a resolver) names the application can't find.execute: each node once, in order. Fan-out: a node's frame goes to every node after it. Fan-in: a node waits for all its inputs and gets them as(name, frame)pairs. Tees save a node's frame and the data flows on. A node that fails stops the nodes below it (skipped); the others still run.frame: JSON column frames ({model, fields, ids, columns, groups}): build them from rows, union, select columns, filter rows, keep the last rows, read CSV, JSON or NDJSON, and print a Markdown table.
The runner reaches the application only through Host: it asks for a node's frame given
its inputs, saves tees, and is told when a node produced a frame or finished. Lookups,
permissions, approvals, storage and every other effect stay in the application.
use ;
use ;
;
let g: = parse.unwrap;
assert!;
let mut app = App;
let run = execute.unwrap;
assert!;
assert_eq!;