# A toy test definition in one grammar file
This is a capsule-form example: one Polydat source that carries its
own parameters, its own description, a reusable model, a test flow
bound as a comprehension, and a traversal that unwinds a hierarchic
dataset and derives the load, read, and verify statements of that flow
in every activation. Every row and every statement is a pure function
of one coordinate, so any of them can be regenerated on any fiber or
host without state.
The file is
[`examples/toy_test_definition.polydat`](../../examples/toy_test_definition.polydat).
It is driven by the `polydat` binary with no host code. From the repository:
```sh
cargo run -p polydat -- run crates/polydat/examples/toy_test_definition.polydat --cycles 2 --emit map
```
## The grammar
```polydat
// Toy test definition: a self-describing, hierarchic, parameterized
// dataset and test flow in one Polydat grammar file.
//
// Hierarchy: cycle -> row -> (tenant, device, reading)
// Parameters: externs with defaults; a host may override any of them.
// Traversal: a comprehension over phases, intervals, and partitions of
// the row domain, bound as a producer and traversed below.
// Flow: schema, load, read, and verify statements derived from
// the same coordinate, so any row can be regenerated. The
// load statement carries a JSON document rendered by a tile.
// ---- Coordinates and parameters ---------------------------------------
// The root coordinate. Each traversal activation below has its own
// `cycle`, local to its slice; this one positions the root scope.
input cycle: u64
// Runtime parameters are externs with defaults; a host may assign them
// on the command line without recompiling.
extern base_epoch_ms: u64 = 1700000000000
extern rows_total: u64 = 1000000
// ---- Self-description ------------------------------------------------
const dataset := "iot-readings-toy"
const keyspace := "toy"
const table := "readings"
const schema_stmt := "CREATE TABLE {keyspace}.{table} (tenant_id bigint, device_id text, ts bigint, doc text, PRIMARY KEY ((tenant_id, device_id), ts))"
const shape := "tenants=20 devices_per_tenant=50 readings=unbounded"
// ---- Reusable model ---------------------------------------------------
reading_model(seed: u64) -> (temp_c: f64, humidity: f64, status: str) := {
temp_c := normal_sample(input: seed, mean: 21.5, stddev: 2.0)
humidity := uniform_sample(input: hash(seed), min: 30.0, max: 70.0)
status := weighted_strings(hash(hash(seed)), "ok:0.97;degraded:0.02;error:0.01")
}
// ---- Traversal ----------------------------------------------------------
// The test flow is a comprehension bound as a producer: every phase runs
// over every quarter of the row domain, at two reading intervals. Its
// element names are wires inside the traversal below.
flow := for phase in load,verify, interval_ms in 1000,60000, p in partitions("*/4", {rows_total})
for flow {
// Each activation owns the slice `p` of the row domain. The cursor is
// narrowed to `p` at activation, and the local cycle maps onto an
// absolute row ordinal inside the slice.
cursor rows = range(0, rows_total) over p
row := mod_in(cycle, rows.cursor)
// Structural parameters are literal constants in the decomposition.
// The first two dimensions are bounded; the third is unbounded, so
// the dataset grows with the row count.
(tenant, device, reading) := mixed_radix(row, 20, 50, 0)
// ---- Entities -----------------------------------------------------
tenant_id := hashed_id(input: tenant, bound: 1000000)
device_key := interleave(tenant, device)
device_id := hashed_uuid(device_key)
device_kind := weighted_strings(device_key, "sensor:0.7;gateway:0.2;controller:0.1")
reading_seed := hash(interleave(device_key, reading))
(temp_c, humidity, status) := reading_model(reading_seed)
ts := base_epoch_ms + reading * interval_ms
// A coarse health signal derived from the same coordinate: nonzero
// when the reading should be flagged by a verifier.
flagged := if temp_c > 26.0 { 1 } else { 0 }
// ---- Document -----------------------------------------------------
// The reading as a JSON document. A tile is a template whose holes
// are wires: numbers render bare and strings quoted by their types,
// the `meta` arm is one static copy, and `samples` repeats its body
// over a comprehension. The tile is a wire like any other.
tile doc : json := {
"meta": { "schema": 3, "source": "polydat", "units": { "temp": "C", "rh": "%" } },
"tenant": ${tenant_id},
"device": ${device_id},
"kind": ${device_kind},
"ts": ${ts},
"reading": { "temp": ${temp_c | .2}, "rh": ${humidity | .1}, "status": ${status} },
"samples": [ @for s in 0..4 { { "n": ${s}, "temp": ${temp_c + s | .2} } } ],
"flagged": ${flagged: bool}
}
// ---- Test flow ----------------------------------------------------
// The load statement carries the document raw: it is already JSON.
tile load := <<<
INSERT INTO ${keyspace}.${table} (tenant_id, device_id, ts, doc) VALUES (${tenant_id}, '${device_id}', ${ts}, '${doc!}')
>>>
read_stmt := "SELECT doc FROM {keyspace}.{table} WHERE tenant_id = {tenant_id} AND device_id = '{device_id}' AND ts = {ts}"
verify_stmt := "expect temp_c = {temp_c}, humidity = {humidity}, status = '{status}'"
// The statement this activation executes is selected by its phase.
stmt := select_str(str_eq(phase, "load"), load, verify_stmt)
}
```
## What each part does
| `input cycle: u64` | Root coordinate | Positions the root scope. Each activation has its own local `cycle`. |
| `extern ... = default` | Runtime parameters | Typed slots with defaults. A bare `name=value` argument to the binary rewrites the default before compilation. |
| `const ... :=` | Self-description | Values computed once at scope init: the dataset name, keyspace, table, schema statement, and the shape of the coordinate space. |
| `reading_model(...) := { ... }` | Module | A typed, reusable computation with three outputs, resolved from the same file. |
| `flow := for ...` | Producer | Binds the test flow as a comprehension value: two phases, two intervals, and four partitions of the row domain. `{rows_total}` reads the extern when the traversal opens. |
| `for flow { ... }` | Traversal | Compiles the body once into a child program and activates it once per tuple, with `phase`, `interval_ms`, and `p` as typed wires and `base_epoch_ms`, `keyspace`, `table`, and `rows_total` cascaded from the root. |
| `cursor rows = ... over p` | Partitioned cursor | Narrowed to the activation's partition. Its slice sets how many cycles the activation runs. |
| `mod_in(cycle, rows.cursor)` | Slice projection | Maps the activation's local cycle onto an absolute row ordinal inside its slice. |
| `mixed_radix(row, 20, 50, 0)` | Hierarchy | Unwinds one ordinal into tenant, device, and reading. The trailing `0` leaves readings unbounded. |
| `hashed_id`, `normal_sample`, `uniform_sample` | Standard library | Modules from the embedded `.polydat` library, called with named arguments. |
| `"... {expr} ..."` | String interpolation | The schema, read, and verify statements are ordinary bindings that embed typed wires. |
| `tile doc : json := { ... }` | Tile | The reading as a JSON document. Holes are wires: `u64` and `f64` render bare, `Str` quoted, `${flagged: bool}` as a boolean; `\| .2` is a format; the `meta` arm is one static copy; `@for s in 0..4` repeats its body over a comprehension. |
| `tile load := <<< ... >>>` | Tile carrying a tile | The load statement, with the document inlined raw through `${doc!}`. Both tiles are wires like any other. |
| `select_str(str_eq(phase, "load"), ...)` | Phase selection | The statement an activation executes follows its `phase` element. |
## Running it
The binary plays the host. It compiles the file, resolving the in-file
module from the file's own directory, and runs it in traversal mode:
the emit binding is inserted into the `for` body, the sixteen
activations are taken by index across fibers, and each activation runs
its cycles under the traversal rule, capped by `--cycles`. Externs are
assigned with bare `name=value` arguments.
Two cycles per activation, showing the hierarchy each slice starts from:
```text
$ polydat run toy_test_definition.polydat --cycles 2 --emit map \
--outputs phase,interval_ms,row,tenant,device,reading,ts,status
traversal 0: for flow (16 activations)
phase=load interval_ms=1000 row=0 tenant=0 device=0 reading=0 ts=1700000000000 status=ok
phase=load interval_ms=1000 row=1 tenant=1 device=0 reading=0 ts=1700000000000 status=ok
phase=load interval_ms=1000 row=250000 tenant=0 device=0 reading=250 ts=1700000250000 status=ok
phase=load interval_ms=1000 row=250001 tenant=1 device=0 reading=250 ts=1700000250000 status=ok
phase=load interval_ms=1000 row=500000 tenant=0 device=0 reading=500 ts=1700000500000 status=ok
phase=load interval_ms=1000 row=500001 tenant=1 device=0 reading=500 ts=1700000500000 status=ok
```
The document each reading carries, one per activation with one cycle
each. Naming the tile in `--emit` writes its rendered text and nothing
else; the first of the sixteen:
```text
$ polydat run toy_test_definition.polydat --cycles 1 --emit tile:doc -q
{
"meta": { "schema": 3, "source": "polydat", "units": { "temp": "C", "rh": "%" } },
"tenant": 607535,
"device": "d9ac876f-bb3a-4bc7-b9f8-382893178079",
"kind": "sensor",
"ts": 1700000000000,
"reading": { "temp": 22.28, "rh": 35.5, "status": "ok" },
"samples": [ { "n": 0, "temp": 22.28 },{ "n": 1, "temp": 23.28 },{ "n": 2, "temp": 24.28 },{ "n": 3, "temp": 25.28 } ],
"flagged": false
}
```
The load statement carries that document inline. One activation's
statement, as a CSV row:
```text
$ polydat run toy_test_definition.polydat --cycles 1 --emit csv --outputs phase,stmt -q
phase,stmt
load,"INSERT INTO toy.readings (tenant_id, device_id, ts, doc) VALUES (607535, 'd9ac876f-bb3a-4bc7-b9f8-382893178079', 1700000000000, '{
""meta"": { ""schema"": 3, ""source"": ""polydat"", ""units"": { ""temp"": ""C"", ""rh"": ""%"" } },
...
""flagged"": false
}')"
```
Assigning the externs reshapes the run without recompiling anything but
the two rewritten declarations:
```text
$ polydat run toy_test_definition.polydat base_epoch_ms=5 rows_total=400 --cycles 1 --emit map --outputs row,ts -q
row=0 ts=5
row=100 ts=5
row=200 ts=5
row=300 ts=5
```
`polydat check --stats` reports two programs, the root and the one
traversal body; `polydat explain traversals` prints the body's elements
and cascade, and `polydat explain tiles` prints each tile's skeleton and
how every hole was typed and encoded.
## Reading the output
- **Each activation starts at its own slice.** The traversal's tuples
are ordered by phase, then interval, then partition, so the first four
activations are the four quarters of the row domain in the load phase
at a one-second interval. Row 250000 is reading 250 of device 0 in
tenant 0, because the 1000-row stride of the first two dimensions has
been walked 250 times by the ordinals below the slice. Slices never
overlap.
- **Identity is stable across slices.** All four quarters land on tenant
0 and device 0 for their first row and get the same tenant id and
device uuid. Only the reading and its derived values differ.
- **Elements drive the body.** The fifth activation is the same load
phase and first quarter at a one-minute interval; `ts` would step by
60000 between its readings. The verify-phase activations select the
verify statement through `phase`.
- **Externs reach the traversal.** `rows_total` sizes both the
partition spec and the cursor, so assigning it to 400 makes each
quarter 100 rows. `base_epoch_ms` cascades into the body as the
timestamp base.
- **The document is a wire.** `doc` renders once per cycle from the
same coordinate as everything else, so the load statement, the
verify statement, and the document can never disagree about a
reading. `${temp_c | .2}` and `${humidity | .1}` fix the precision
the document carries; the verify statement keeps the full values.
## What this shows and what it does not
The whole flow is declared in the grammar: the comprehension, the
partitioning, the per-activation cursor, and the statements. Polydat
compiles the body once and the binary activates it sixteen times; the
build counter stays flat while it does. What the file does not decide is
scheduling: how many fibers, how many cycles per activation, and what to
do with each statement are the host's choices, and the binary exposes
them as options. The contract is [The `for` Construct](../design/for_traversal.md);
the document tiles are [Polytile](../design/polytile.md), walked through in
[the Polytile tutorial](polytile_tutorial.md); each feature is shown on
its own in [Illustrations](illustrations.md).