arblang 0.1.24

Visualize and modify Unix pipelines: pipe a stream in, arb spawns a dynamic TUI
Documentation
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
# arb — SPEC

**arb** is a standalone, original language on **fusevm/JIT** for **visualizing and modifying Unix pipelines**: drop it in a pipe and it spawns a **dynamic TUI (ratatui) or served web page (zgui components)** built from a declarative spec. It is a **jq/xpath/css/yq superset** — every leg machine-checked against its own reference tool, with the per-leg containment measured in §8 — an interactive **megafilter/map** over the live passthrough, its own **Tcl/Tk-flavored DSL**, and a **preset library / package manager** so users share dashboards — *a TUI for every pipeline*. (LSP/DAP stdio frontends ship. Akka-style actors ship: `actor NAME(state) { on MSG { … } }` + a `via NAME * N` pipeline op that fans the stream across a supervised worker pool in parallel — see §15.)

Original language (stryke's class), **not a port**. MIT, standalone crate, lean (rubyrs-scale, not stryke-scale).

---

## 0. Positioning

- **The synthesis + ecosystem**, not any single leg. Prior art per leg: Tcl'88, Tk'88, Expect'90 (spawn/react), dasel (unified query), ratatui (TUI), Streamlit/Textual-serve (served web UI), filt (interactive pipe grep — *single filter box, filter-only, TUI-only; not comparable*). No tool is a pipe-native, dual-target (terminal+web), component-generating UI language with a shareable dashboard registry.
- **Not a server-only thing**: terminal-invoked, pipe-driven. Web target spawns a local UI host (like `textual serve`), not a daemon.

## 1. Invocation

```sh
cmd | arb dash.arb            # TUI (ratatui)
cmd | arb -t web dash.arb     # served zgui page + WS live update
cmd | arb -p http             # preset (== import http)
cmd | arb                     # zero-config: sniff input/upstream cmd → auto preset
cmd | arb -e 'text .t <- in'  # inline
producer | arb dash.arb | consumer   # mid-pipe: controls shape downstream output
arb -l                        # list presets/packages
arb save dash.arb as api      # register a user preset
```

**fzf drop-in.** `arb --fzf` resolves its options the way the `fzf` binary does —
`$FZF_DEFAULT_OPTS_FILE`, then `$FZF_DEFAULT_OPTS`, then the command line, later
winning — and honors the presentation set (`--layout`/`--reverse`, `--border`,
`--info`, `--color`, `--pointer`, `--marker`, `--ellipsis`, `--scrollbar`,
`--scroll-off`, `--cycle`, `--tac`, `--tiebreak`, `--bind`, `--min-height`,
`--ansi`) on top of the matching and I/O set (`--exact`, `--no-sort`/`--sort`,
`-q`/`--query`, `-i`/`--ignore-case`, `+i`/`--no-ignore-case`, `--smart-case`,
`--multi`, `--prompt`, `--header`, `--header-lines`, `--height`, `--preview`,
`-f`/`--filter`, `-d`/`--delimiter`, `-n`/`--nth`, `--with-nth`,
`--print-query`, `--expect`, `--with-shell`). Child processes (a `--preview`
command) start with `$SHELL -c` like fzf's, not `sh -c`. A repeated option takes
its LAST value, as fzf does — `$FZF_DEFAULT_OPTS` and the call site both setting `--prompt` is the
normal case, not an error. The whole fzf 0.74 option surface parses, including
every `--no-X` negation, which cancels an earlier `--X` rather than being
dropped — `--preview CMD --no-preview` leaves no preview — and an
optional-value option consumes the next word only under fzf's own rule for it
(a word that isn't a flag for `--border`, a number for `--gap`). Matching
itself is fzf's algorithm, ported from `src/algo/algo.go` into `src/algo.rs`
(`FuzzyMatchV2` + `ExactMatchNaive`), so rankings are identical line for line —
verifiable with `--filter`, which prints the ranked matches without a UI.
`--height 100%` is fzf's full-screen form and runs on the alternate screen, so
quitting restores the terminal untouched; a smaller `--height` draws below the
cursor and scrolls the terminal to make room, sized like fzf's (percentages get
the `--min-height=10+` floor); arb places that viewport itself rather than
asking crossterm, whose cursor query writes to stdout — arb's data channel — and
never returns when stdout is a pipe. The picker it draws — layout, separator,
pointer/marker glyphs, palette, scrollbar, ellipsis, scroll margin, ranking —
matches what `fzf` draws for the same environment, so `ZPWR_FZF='arb --fzf'`
changes nothing on screen. Flags with no arb analog are accepted and ignored, and
a `--bind` naming an unimplemented action is skipped rather than half-applied.

## 2. Lexical (Tcl-flavored, NOT Tcl)

```
verb arg arg { block }     # command + args; {} is a verbatim block
# comment                    (no $, no [cmd] subst, no expr{})
;                          # optional stmt separator (newline also separates)
.a.b.c                     # widget path (dot hierarchy, Tk-style)
```

## 3. Values

```
42  3.14  "s"  's'  true  false  nil
1s 500ms 2m        # durations
1kb 4mb            # sizes
/re/  /re/i        # regex
[1,2,3]            # list
{a:1, b:2}         # map
```

## 4. Variables (Python/Swift-lite)

```
max = 100          # immutable
var n = 0          # mutable
n = n + 1
```

## 5. Functions & lambdas

```
fn pct(v, m): v / m * 100          # single-expr body
fn norm(xs) { hi = max(xs); map(x => x/hi, xs) }   # block; last expr returns
dbl = x => x * 2                   # lambda
add = (a, b) => a + b
pct(50, 200)                       # call
```

## 6. Operators

```
+ - * / %      arithmetic
== != < <= > >= comparison
and or not     logical
+              string concat
x matches /re/ regex test
x in [..]      membership
a..b           range
( … )          grouping — a full expression, including a comparison, a
               logical, or a `?:` (`(x > 1) and (x < 5)`, `not (a or b)`)
|>             value pipe: xs |> filter(even) |> sum
```

Every value is an f64, so a predicate has a NUMBER for its value: `1` when it
holds and `0` when it does not, for `==`/`<`/`and`/`or`/`not`/`in` alike. That
matters wherever an expression's value is printed rather than tested —
`map x > 1 or x > 2` emits `1`, never a count of how many sides held.

A number literal may carry an exponent (`1e17`, `2e+2`, `1.5e-6`, `1E3`), which
is also how arb prints one back.

### How a computed number prints

A computed value renders the way `jq -r` renders a computed double: the shortest
digits that round-trip and, among those, the ones NEAREST the value, positional
up to the point where jq switches, then exponential with a signed
two-digit-minimum exponent.

```
1e15 * 1   -> 1000000000000000       1e16 * 1  -> 1e+16
1.5e16 * 1 -> 15000000000000000      1.5e17 * 1 -> 1.5e+17
0.0001 * 1 -> 0.0001                 0.00001 * 1 -> 1e-05
1e18 / 3   -> 333333333333333300     2e-9 * 2  -> 4e-09
```

The switch is on the DIGIT COUNT, not the magnitude — `1e16` is exponential and
`1.5e16`, one significant digit longer, is not.

Shortest and shortest-AND-CLOSEST are different rules, and the difference shows.
Where neighbouring doubles sit further apart than one unit in the last decimal
place, several decimals of the shortest length parse back to the same double, and
only one of them is nearest. `191510495617760.12 * 1` prints those digits because
they are the nearest; `…13` round-trips just as well and is what stopping at the
first round-tripping candidate gives. The ties bunch around 1e14..1e17, seldom
enough that a pool spread evenly across the exponent range almost never lands on
one.

A result with no finite value follows jq as well: an infinity CLAMPS to ±DBL_MAX
(`1e308 * 2` -> `1.7976931348623157e+308`) and a NaN prints as `null`
(`1e308 * 2 * 0`). Neither has a spelling in JSON.

One deliberate difference from jq remains:

- **There is no literal preservation.** A number jq never computes on is held as
  a DECIMAL rather than a double and reprinted from that, so `.n` on
  `{"n":1e17}` gives `1E+17` — and unary `-.` preserves it too, printing `-1E-7`
  where the computed path gives `-1e-07`. It is not the source text being echoed:
  `1e17`, `1E17` and `1e+17` all come back as `1E+17`, decNumber's canonical
  form.

  Matching it is not a formatter change. That decimal holds values no double can
  — `{"n":1e400}` reprints as `1E+400`, and a 30-digit integer keeps all 30
  digits, where an f64 overflows and rounds respectively. Following jq here would
  mean carrying an arbitrary-precision decimal beside every number, which is
  exactly the "every value is an f64" rule above. So arb prints the double, and
  `scripts/expr_paths.sh` reports the difference rather than hiding it.

### `%` takes a remainder, it does not truncate

`%` is the f64 remainder of the two operands: `5.5 % 3` is `2.5`. jq truncates
both operands to integers first and answers `2`. The two agree on every integer
input, which is why both harnesses scored parity for `%` until their corpora grew
a fractional value. The probes for it are red, on purpose, and stay that way
rather than being reworded to match either side.

For arb's own expressions (`map x % 3`) the remainder is the settled rule: every
value here is an f64, and truncating the operands would be the one place the
language stopped believing that.

What is NOT settled is the jq context. `%` also reaches the jq front-end — a
leading `.` routes the whole body there — and lowers to the same fusevm op, so
`out { in.json; .n % 3 }` on `{"n":5.5}` answers `2.5` where jq answers `2`. That
front-end promises jq's answers, and arb already settles this exact kind of
collision by CONTEXT rather than by picking one meaning globally: `flatten` and
`to_entries` answer as jq while the native `entries` keeps its line-per-key
shape, under distinct spellings. Whether `%` should join them is open;
`scripts/jq_parity.sh` probes it so the difference is reported either way.

## 7. Pipe & sources

```
in                       stdin (lines)
in.json in.xml in.html in.yaml in.toml in.csv   parsed stream
out.yaml out.json out.props out.xml out.csv out.tsv [INDENT]
                         re-serialize the stream (yq's -o= / -I) ✅
spawn ps aux             arb launches a process; its stdout is the stream ✅
spawn { tail -f a.log; grep err }   block form → one `sh -c` string ✅
! vmstat 1 every 1s      re-run a command on a timer ✅ (headless runs it once)
< file.log               read a file as the stream ✅ (quote absolute paths: `< "/var/log/x"`)
out { … }                downstream emission (to | next), shaped by controls
spawn -pty CMD           run CMD on a pseudo-terminal (acts interactive) ✅
send "text"              write to a `spawn -pty` child's stdin (Expect) ✅
```

`spawn CMD` (or `spawn { CMD }`) makes a spec self-sourcing: arb runs CMD via
`sh -c` and feeds its stdout into the stream in place of stdin (fire-and-forget;
the child dies with arb). `< FILE` reads a file as the stream instead (folded to
`cat -- FILE`). `! CMD every Ns` re-runs CMD on a timer, feeding each run's
output into the stream — in the TUI/served dashboard it loops in the background;
headless (piped onward) it runs CMD **once**, because a reducer/emit over an
endless timer source could never terminate. At most one stream source may be
declared (`spawn`/`< FILE`/`! …`); a CLI `--run 'PROD | _ | CONS'` producer wins
over all of them. **`spawn -pty CMD`** runs the spawn command on a pseudo-terminal
so it behaves as if interactive, and keeps a writer to its stdin so a **`send
"text"`** action (a bind/expect action, like `set`/`beep`/`exec`) can drive it —
Expect-style automation, e.g. `expect { /password:/ send "hunter2\n" }`. The
`send` write happens in the TUI (headless falls back to a plain pipe). The
`sel` selection widget (§9) exposes a widget's highlighted row as `.<path>.sel`
for a `send`/`where`/`tell` to consume.

## 8. Query — jq/xpath/css/yq superset (uniform over all formats)

```
field NAME        key (jq .name); field a b c = a.b.c; field N = Nth ws column
fields N M …     project/reorder whitespace columns (1-based): fields 1 3 -> cols 1 and 3
each              iterate (jq [])
find TAG          recursive descent (xpath //)
attr NAME         attribute (xpath @, css)
sel { CSS }       CSS selector (html)
where(PRED)       filter (jq select)
pick a b c        project object to keys (jq {a,b,c}); keeps listed order
b64             base64-encode each line
b64d            base64-decode each line (invalid passes through)
hex             lowercase hex-encode each line (byte-wise)
unhex           hex-decode each line to UTF-8 (invalid passes through)
urlenc          percent-encode each line
urldec          percent-decode each line
extract /re/    first regex match per line (capture group 1 if any); no-match dropped
split DELIM     explode each line by DELIM into one line per part
substr A B      character substring [A,B) 0-based, clamped
chars           explode each line into one char per line
title           title-case each line
repeat N        repeat each line's content N times
set K V         set json object key K to string V
del K           remove json object key K (jq del)
rename OLD NEW  rename json object key OLD to NEW
default K V     set json object key K to V only if absent
merge           merge all json objects into one (later keys win)
floor           floor each numeric line
ceil            ceil each numeric line
clamp LO HI     clamp each numeric line into [LO,HI]
abs             absolute value of each numeric line
round           round each numeric line to the nearest integer
commafy         thousands-group each numeric line (1234567 -> 1,234,567)
bytes           humanize a byte count, 1024-based (1536 -> 1.5 KB)
duration        humanize seconds as the two largest units (3661 -> 1h 1m)

delta           consecutive differences of the numeric series (n -> n-1) — a counter's rate-of-change
cumsum          running (cumulative) total of the numeric series
sma N           trailing simple moving average, window N (length-preserving; smooths a noisy series)
ewma A          exponentially-weighted moving average, smoothing factor A in (0,1] (s0=x0)

median          median of numeric lines (reducer)
stddev          population standard deviation (reducer)
percentile N    Nth percentile 0-100, nearest-rank (reducer); p50 p90 p95 p99 are sugar
range           max minus min of numeric lines (reducer)
product         product of numeric lines (reducer)
bins N          bucket numeric lines into N equal-width bins -> (label -> count) pairs

apply .name     splice an `input .name` widget's live value in as a sub-pipeline (megafilter/map)

sort_by F   stable-sort json records by field F (numeric when all values parse, else lexical; non-objects last)
unique_by F   keep first JSON record per distinct value of field F (dedup by F)
count_by F    count json records grouped by field F (value -> count, count desc)
min_by F      return the JSON record whose numeric field F is smallest
max_by F     emit the record with the largest numeric field F (reducer)
has KEY          keep only JSON-object lines that contain key KEY
entries          jq to_entries: emit {"key":k,"value":v} per key of each JSON object line
flatten          flatten a JSON array to its leaves, ALL levels (jq flatten), one leaf per line
add               jq add: sum a numeric JSON-array line, concat a string array, [] -> ""
over N          keep numeric lines strictly greater than N (drops non-numeric)
under N            keep numeric lines strictly less than N
between A B   keep numeric lines x with A <= x <= B (inclusive), drop the rest
enumerate         prefix each line with its 1-based index and a tab
words                split each line on whitespace into one word per line (flatten)
dedup                collapse adjacent duplicate lines to one (classic uniq)
tailn N       keep the last N lines (complement of take)
pad N            right-pad each line with spaces to a minimum width N (no truncation)
lpad N          left-pad each line with spaces to minimum width N
grepf FIELD /re/   keep lines whose FIELD (json key or 1-based ws column) matches /re/
flip            reverse the characters of each line (Unicode scalar reversal)

names  vals       native line-per-key / line-per-value (jq's `keys` and
                  `values` keep their own spellings and their own shapes)
map(FN)           jq map: `[.[] | FN]` — returns one ARRAY line, scoped per input
count sum min max avg tally    aggregates
sort sort_by(FN) group_by(FN) uniq
over N  under N   numeric threshold
index N  slice A B  positional
```

| jq/xpath/css | arb |
|---|---|
| `.users[].name` | `field users; each; field name` |
| `.items[] \| select(.price>10)` | `field items; each; where(price>10)` |
| `//a/@href` | `find a; attr href` |
| `div.card h2` | `sel {div.card h2}` |

**Literal front-ends.** The right column is arb's native form, but inside a
`source { … }` / `out { … }` body you may write the **jq** or **xpath** literal
directly — it compiles to the same ops:

```
out { in.json; .users[] | select(.age >= 18) | .name }   # jq: path, iterate, filter
out { in.json; map(.price); add }                        # jq map() -> one array, then a native verb
out { in.html; //a/@href }                               # xpath: descendant + attribute
out { in.html; //div[@class]//span/text() }              # xpath: predicate + child + text()
```

**The whole jq language.** arb implements jq, not a subset of it: identity `.`,
key/path `.foo.bar`, iterate `.[]`/`.foo[]`, index `.[N]` (and negative `.[-1]`),
slice `.[a:b]`, pipe `|`, the comma operator `.a, .b`, object and array
construction (`{a: .a}`, `[.a, .b]`), parentheses, `if`/`then`/`elif`/`else`,
`try`/`catch` and the `?` suppressor, the `//` alternative, `..` recursive
descent, `as` bindings including array/object destructuring and the `?//`
alternative, `reduce` and `foreach`, `label`/`break`, `def` with filter and value
parameters (and recursion), `$ENV`/`$__loc__`, string interpolation `\(…)` and
every `@format`, the whole assignment family (`=`, `|=`, `+=`, `-=`, `*=`, `/=`,
`%=`, `//=`), path expressions (`path`, `paths`, `getpath`, `setpath`,
`delpaths`, `del`, `pick`, `to_entries`/`from_entries`/`with_entries`), the regex
family (`test`/`match`/`capture`/`scan`/`split`/`splits`/`sub`/`gsub`), the
stream builtins (`tostream`/`fromstream`/`truncate_stream`, `input`/`inputs`,
`limit`/`first`/`last`/`nth`/`until`/`while`/`repeat`/`isempty`), and the
date and libm surfaces.

That is not a list to be taken on trust. `scripts/jq_parity.sh` carries a
`superset_probe` that compares jq's own `builtins` output against arb's and fails
on any `name/arity` jq defines and arb does not; it currently reports none
missing. On its first run it reported 44, which is how `JOIN`, `bsearch`, `skip`,
`toboolean`, `trimstr`, `format` and the libm names from `acosh` to `yn` came to
be implemented. Every other construct is byte-diffed against `jq -rc` on the same
input, and a construct applied to the WRONG TYPE must be refused by both engines
— with jq's refusal checked, so a refusal arb invented alone is never scored as
parity.

### Containment is measured per leg

One containment probe per reference, each enumerating that reference's own
surface and reporting what arb is missing. Current run: **825 probes, 825 pass,
0 diverged, no allowlist.** The round-4 jq additions (`repeat`, positional
`sub`/`gsub` generators, code-point `indices`, the `s`/`p` regex flags, UTC
`strftime` zones, string repetition, and jq's fixed refusal wordings) were
measured on a machine without `yq`: 728 pass, 0 diverged, 159 yq probes skipped
(703 pass / 24 diverged on the same corpus before the fixes). The round-5 additions
(value parameters that yield several values run once per combination in jq's
order, the translator's `.[]`/`map`/`to_entries`/`flatten`/`add` keeping key
order and number literals, literal-preserving negation, empty regex matches after
a match, `{$name: P}` patterns, and jq's refusals for `unique`/`min`/`max`/
`*_by`/`reverse`/`input`/non-path calls, and `fromjson` as jq's own `jv_parse.c`
reader — what it accepts and its positioned refusals, and jq 1.8's decNumber
number model: two literals compare as decimals, a computed `-0` keeps its sign,
`have_decnum` is true) measured 759 pass, 0 diverged, 159 skipped (729 pass /
30 diverged on the same corpus before the fixes). The round-6 additions (`as`
binding jq 1.8's whole expression, a `def` after a pipe, literal- and
negation-led programs, a trailing comma in an object, quoted subscripts that
generate or interpolate, `strptime`/`mktime`/`gmtime`/`strftime %s` as jq's C,
`in.json` reading JSON texts rather than lines, and `input_line_number` over the
whole stream) measured 772 pass, 0 diverged, 159 skipped (760 pass / 12 diverged
on the same corpus before the fixes). The round-7 additions (array assignment's
NaN / too-large index refusals, `contains` comparing jv kinds only at the top
level, `f_match`'s argument wordings, `%` with NaN or a `-1` divisor, object-key
slices as `parse_slice` reads them, `escape_string`'s `\0` for NUL in
`@csv`/`@tsv`/`@html`/`@sh`, `range`/`limit`/`skip` as the RANGE opcode and
`builtin.jq` define them, and `nearbyint` rounding half to even) measured 797
pass, 0 diverged, 159 skipped (774 pass / 23 diverged on the same corpus before
the fixes). The round-8 additions (`try`/`catch` bodies that are negations,
`{$v: e}` keying by the value of `$v`, an empty array pattern refused, `builtins`
without `_` internals, `delpaths` as `jv_dels`, `?` after an index suppressing only
that step, jq's `path_intact` tracking through `label`/`reduce`/`foreach`/`try` with
its "near attempt to …" refusals, `jv_dump_string_trunc` byte truncation, input
lines read by the `jv_parse.c` port (`NaN`, `Infinity`, `.5`, BOM), `implode` of
NaN, `jv_get`/`jv_set` refusals, `length` keeping a number literal, `f_match`'s
empty-capture key order, and `_strindices` wordings) measured 816 pass, 0
diverged, 159 skipped (797 pass / 19 diverged on the same corpus before the
fixes). The seeded differential fuzz rounds that followed (they are a `cargo test`
of their own, not part of that script's corpus) moved the regex family onto
Oniguruma — the engine jq links, in its Perl_NG syntax, so look-around,
backreferences, `\K`, `\h`, the modifier letters and the engine's own refusal
text are jq's — and followed jq where its bytecode shows through: `path_intact`
tracking in destructuring patterns, `and`/`or` and the right-hand side of an
arithmetic assignment, `walk`/`map_values`/`from_entries`/`sub`/`gsub`/`splits`
as jq's own definitions, a parsed number never being `jv_identical` to one a
builtin computed from it, an update stopping at its first output, `?//`
catching what is raised downstream of it (a `break` included), `reduce`
re-entered over a `null` input, and `break $l` as the error jq compiles it to
(`$l | error`, the label value being `{"__jq": n}`): a `try`/`?` between a
`break` and its label catches it with that object, a handler that re-raises it
still ends the label, and `n` counts every label the run entered, builtin.jq's
own included, and input string bodies decoded as `jv_parse.c` decodes them: a
high surrogate with no low one after it, a non-hex `\u` body and a raw control
byte make the line non-JSON (so it keeps the text reading) instead of a repaired
value jq never produces. fixes). The 825 above is the last run WITH `yq`; it was not re-measured since.

| leg | reference | containment | status |
|---|---|---|---|
| jq | `jq` 1.8.2 `builtins` | every name present | **superset** |
| xpath | XPath 1.0 (W3C REC) via `xmllint` | **48 of 48 enumerated constructs present** | **superset** of the enumerated surface |
| css | Selectors 3/4 via `scraper` | 24 selection probes match the equivalent XPath | matches the reference on every probe |
| yq | mikefarah/yq v4.53.6 | every enumerated operator present | **superset** of the enumerated surface |

**XPath.** The subset this section used to document — `//`, `/`, `@`, `[@a]`,
`[@a='v']`, `[contains(@a,'v')]` — was XPath-SHAPED syntax compiled to a CSS
selector, and the probe found 46 of 48 constructs missing. `src/xpath.rs` is a
real engine now: `xpath_syntax.rs` lexes and parses the grammar of the W3C
Recommendation, `xpath_eval.rs` evaluates it over the parsed document. All 13
axes in explicit `axis::test` syntax, the node tests `node()`/`text()`/
`comment()`/`processing-instruction()` plus name and `*` wildcard tests,
predicates with correct proximity positions (§2.4 — and a predicate on a REVERSE
axis counts along that axis, so `preceding-sibling::p[1]` is the NEAREST
preceding sibling), the 27-function core library (§4), and the comparison,
arithmetic and union operators with §3.4's existential node-set semantics.
**59 `xp_probe`s** byte-diff the engine against `xmllint --html --xpath`.

**The four silent wrong answers are gone.** They were the reason this leg was
the priority: the promise below — that anything outside the subset is a hard
error, "never silently reinterpreted" — did not hold for them.
`[@a='x' or @a='y']` and a chained predicate `[@a='x'][@b='y']` both exited 0
with an EMPTY selection where XPath selects nodes, and a ROOTED path
(`/li/text()`, `/div/h2/text()`) exited 0 with a NON-EMPTY node set where XPath
selects nothing, because a leading `/step` was compiled as a descendant match.
A real engine removes the whole class by construction; `tests/xpath.rs` pins
each of the four against xmllint's own answer.

**The data model (§5) is reconciled with html5ever explicitly.** A doctype is
not one of §5's seven node types, so `count(//node())` must not see it. And
html5ever implements the HTML5 tree-construction algorithm, which ALWAYS creates
a `head` and a table `tbody`, where libxml2 creates each only when the source
has one — measured: `xmllint --html --xpath 'count(//head)'` answers 0 for
`<a>X</a><p>P</p>` and 1 for markup with a real `<head>`, and `count(//tbody)`
answers 0 for `<table><tr>…` and 1 when `<tbody>` is written. Both are hidden
from the data model AND from the serialization, so `//*`, `//node()` and every
positional predicate over them agree with the reference.

**Two stated boundaries, neither silent.** At arb's COMMAND position an
expression must carry a character only XPath uses (`/`, `@`, `::`, `(`, `[`,
`|`): a bare NCName is a legal relative location path, so `bogus` would parse as
`child::bogus`, select nothing and exit 0 — the silent answer this engine exists
to remove, reintroduced through the front door. It stays an `unknown verb`
diagnostic, and the step is spelled `//bogus`, `./bogus` or `child::bogus`.
And `contains` is the one core-function spelling jq already answers, so at
command position it stays jq's; XPath's two-argument form is reachable inside a
predicate, where it is probed. Both are the `keys`/`names` rule again — a
language does not get to claim a spelling another meaning already owns.

**CSS.** `sel` hands its selector to `scraper`. Both gaps this section used to
list are closed and probed. A `Selector::parse` failure is a hard error at BUILD
time naming the selector, where it used to map to "no matches" — so a MALFORMED
selector was indistinguishable from one that legitimately selects nothing, the
same silent-answer failure ruled out above (5 `css_bad` probes). And an
attribute value may be written with either quote: arb's command lexer turns
`"…"` into a string argument, and the reconstruction re-quotes it, so
`a[href="/x"]` no longer reaches the parser as `a[href=/x]`.

**yq.** `src/xpath.rs`'s problem was a missing engine; this leg's was a missing
VALUE MODEL. `yq_superset_probe` found **61 operators missing**, and among them
every node-metadata accessor — `anchor`, `alias`, `tag`, `style`, the three
comment positions, `key`, `is_key`, `path`, `parent`, `line`, `column`, `kind`.
Those are not builtins anyone forgot. They read metadata a YAML node carries and
a jq value has no slot for, which is the reason yq exists over jq at all, and
they cannot be added one at a time.

So the metadata rides ALONGSIDE the value, in the shape `JqVal::Num(f64,
Option<literal>)` already used for a number's source text — same problem, same
answer, already in the tree. `JqVal::Node` boxes a value with its
`crate::ynode::NodeMeta`, `bare()` unwraps at every operation whose answer is
about the value, and a node with nothing to record is not boxed at all. The
load-bearing property is that the JSON reader cannot construct the variant: only
`crate::yaml` can, so a JSON program reaches every answer through exactly the
arms it reached them through before, and **the jq leg is untouched by
construction rather than by inspection**.

Two things the parser does not hand over are recovered from the SPANS it does. A
`#` inside a scalar's span is content (`pw: "a#b"`, a comment-looking line inside
a `|` block) and one outside every span opens a comment; an anchor NAME is read
backwards from its node, over the whitespace and optional tag between them, and
only for a node the parser already reported as anchored. Comment ATTACHMENT
follows six rules measured against `yq v4.53.6` rather than taken from its
documentation — the head comment lands on the KEY node, the line comment on the
VALUE node, and a block followed by a blank line becomes the PREVIOUS entry's
foot.

What the probes check, all byte-diffed and none normalized:

| probe | what it asserts |
|---|---|
| `yq_superset_probe` | every enumerated yq operator exists — **61 missing before, 0 now** |
| `yq_probe` (111) | each operator's ANSWER equals `yq -o=json -I=0`'s on the same node |
| `yq_rt_probe` (14) | `in.yaml; out.yaml` returns the SOURCE FILE byte for byte |
| `yq_norm_probe` (3) | a shape yq does not return either: arb normalizes exactly as yq does |
| `yq_write_probe` (8) | a metadata assignment produces the document yq produces |
| `yq_fmt_probe` (12) | `out.props` is byte-identical to `yq -o=props` |

The round trip is the strongest of the six, and it is asserted against the source
file rather than against yq's output — which is stricter, not looser. `yq '.'` is
not idempotent on every shape: it re-folds a `>` block onto one line, escapes a
non-BMP character as a `"\U0001F680"` sequence, collapses a multi-line flow
collection, reorders `!!map &am` to `&am !!map`, and drops a trailing `...`.
Requiring arb to match yq there would require arb to reproduce yq's own
infidelities. "The file comes back" is the property the claim names, and it
implies matching yq everywhere yq does return the file.

The three shapes yq will not return either are not dropped for that — a shape
nobody checks is the only bad outcome. `yq_norm_probe` asserts them against the
REFERENCE instead, byte for byte and nothing normalized, so arb has to normalize
the same way yq does rather than merely being allowed to differ from the source.
Asserting the weaker of two true properties beats asserting neither.

The fixtures are one per feature the model has to carry — comments in every
position (and on sequence items versus mappings), anchors with aliases and merge
keys, deeply nested anchors, aliases inside merge keys, all six scalar styles,
flow versus block, tags on scalars and on collections, empty values and nulls,
non-ASCII, number spellings, explicit `---`/`...` markers, and a multi-document
stream — so a failure names which one broke.

The reference is asked with `--yaml-fix-merge-anchor-to-spec`. `<<` has two
readings and yq ships both: its default lets a merged key override an explicit
one, and the flag follows the YAML spec. yq's own warning calls the default
"isn't to the yaml spec". arb implements the spec rule, so asking the reference in
its other mode would report a divergence whose content is "arb is right" — which
is not what a divergence should mean. The two modes differ only on merge keys.

`out.FORMAT [INDENT]` is yq's `-o=`/`-I` in arb's spelling (`out.yaml`,
`out.json`, `out.props`, `out.xml`, `out.csv`, `out.tsv`). The DEFAULT rendering
is unchanged: one compact JSON line per document, which is what every existing
pipeline expects.

**What the yq leg does not claim.** ONE spelling is genuinely out, and it is not
a grammar problem. yq's `ref` (`.a ref $x | $x = 5`) binds a MUTABLE HANDLE to a
node, so a later assignment through `$x` edits the document. Nothing in the value
model is mutable — values are `Rc`-shared and copied on write, which is what makes
`reduce`/`foreach`/path updates affordable — so honouring the spelling would mean
a second, mutable value model, and `with(p; f)` already provides the capability.
The other two spellings the docs used to list here are ACCEPTED now: `.a anchor =
"x"` and `.[] as $item ireduce (0; . + $item)` both occupy grammar positions jq
leaves empty, so claiming them cost the jq leg nothing. arb takes yq's spelling
and its own for both.

One behaviour still differs by construction: a node's `path`/`key`/`parent` are
recorded at READ time, so a node relocated by `map`/`pick`/`+` reports where it
was read from, where yq's real parent pointers would follow it. Closing that means
a reference cycle through `Rc` for the whole document, which is a worse trade than
the accessor is worth. (`filename` used to be listed here too; it answers the real
path now when the spec names one with `< FILE`, and `-` for a pipe, which is what
yq answers for a piped document.)

Four YAML inputs are answered where yq ERRORS rather than matched — `.inf`,
`-.inf`, `.nan` and an integer past `i64` — which is the same direction as the
>2^53 deviation above: arb answers, the reference refuses. Being more capable than
the reference is not a containment failure, and none of the four is "fixed" to
match.

**What the node model costs.** Measured on a 20k-record, 2.3MB document against
the same data as JSON, both on the same debug binary: JSON parses in 0.131s and
the same document as YAML in 1.907s, while adding a filter over the result moves
neither (0.183s and 1.904s). So running a query over boxed values costs the same
as over bare ones — `bare()` is free in practice — and the metadata is a tax on
the READ, once, not on every operation. Three suspects were measured inside that
read: formatting a throwaway rendering per scalar was a fifth of it and is gone,
and the per-node path vector and the empty-`Rc` count each moved the total by less
than the run-to-run noise. What remains is saphyr's scanner plus one `Rc<YNode>`
per YAML node.
**What the numbers do not claim.** Containment is measured over the surface each
probe ENUMERATES — for xpath, the 13 axes, 27 core functions and node tests of
the 1999 Recommendation, not XPath 2.0+.

Two engines sit behind that. A jq literal that maps onto arb's line-stream ops
(a path, an iterate, a `select`, a `map`) is translated to them, which is what
keeps arb's own promises about a non-JSON line. Everything else compiles to a
program on arb's jq engine, whose value model is jq's: object keys keep INSERTION
order (`keys_unsorted` and `to_entries` expose it), and a number keeps the source
LITERAL it was read with until arithmetic touches it, printed in decNumber's
canonical form — so `1.50` stays `1.50`, `1e2` prints as `1E+2`, and
`12345678901234567890` round-trips. Two literals compare as the decimals they
spell, whatever their spelling (`3.000000000000000000000001 == 3` is false, and
`sort`/`unique`/`group_by` keep the two apart); a number arithmetic has touched
compares as the double it is, on either side. A literal below the smallest
double is `0.0` carrying the literal's sign, so `-1e-400 * 0` is `-0`.

Names resolve at COMPILE time, as jq's do. A word that is neither an arb verb nor
a jq builtin is still arb's `unknown verb`, not a jq runtime failure — a typo'd
arb verb is far likelier than a jq program, and that diagnostic is what points at
the real mistake.

Supported xpath: **XPath 1.0**, as a real engine rather than a subset — all 13
axes, the four node tests plus name and `*` wildcard tests, predicates with
correct proximity positions, the 27-function core library, and the comparison,
arithmetic and union operators. A standalone `@attr` step is arb's line-stream
continuation (`//a; @href`) rather than XPath's `attribute::` axis from the
document node, which is the one place the pipeline shape shows through: an
expression that is a RELATIVE location path is evaluated per line so it composes
with the step before it, and every other expression is one question about the
document, answered once. A leading `@` belongs to jq instead when it names
one of jq's formats (`@base64`, `@csv`, …) or is followed by a string, which
makes it a format string whatever the name: `@foo "lit"` answers `lit` as jq
does, and `@foo "\(.)"` fails as jq's `foo is not a valid format`. Over
JSON VALUES — after `in.json`/`in.yaml`/`in.toml`/`in.csv`/`in.tsv`, with no
`sel`/`find`/xpath step since — every leading `@` is jq's, so a bare `@foo` (or
`@base32`, which jq 1.8 does not have) raises jq's error instead of selecting no
attribute from a JSON line; after `in.html`, `in`, or an HTML step `@href` is
still the attribute continuation. A body command that opens with a
double-quoted string is jq's too even when jq refuses it: XPath spells a string
literal the same way, so `"\(1;2)"` is jq's syntax error, not the XPath literal
text `\(1;2)`. **Anything
that is not well-formed XPath is a hard error** (`xpath: …`) anchored to the offending verb, raised at BUILD time before
any input is read — never silently reinterpreted.

Where a jq builtin shares a spelling with an arb NATIVE verb (`sort`, `min`,
`max`, `floor`, `abs`, `length`, `add`, `flatten`, `first`, `last`, `split`,
`join`, `del`, `index`, `contains`, `range`, `match`, `repeat`, …) the BARE word
is the native verb and the CALL spelling is jq's — `sort_by v` is arb's,
`sort_by(.v)` is jq's, and `. | sort` reaches jq's. That is the context rule
below, not a jq leak.

**`#` inside `sel { … }` is an ID, not a comment.** `#` opens a comment to
end-of-line wherever a command is expected, and `sel`'s brace used to be re-lexed
as commands — so `sel { #main }`, a spelling the docs advertise, lexed to an
EMPTY block and reported `sel: expected a CSS selector` (exit 1) instead of
selecting the element. `sel` is the one brace in the language whose contents are
not commands (they are a CSS selector), so it is re-lexed with the comment rule
off and `sel { #main }`, `sel { #main p }` and the selector list
`sel { #main p, #two }` all select what xmllint's equivalent XPath selects. Every
other brace holds commands and keeps `#` as a comment.

**`keys` is jq's, and the native verb is `names`.** A native verb is matched
BEFORE the jq fall-through, so a native verb named `keys` shadowed jq's builtin:
bare `keys` printed a line per key where jq prints one sorted array. The context
rule cannot save a spelling jq itself defines — `. | keys` reached jq's answer,
but the bare word could not — so the native line-per-key verb is spelled `names`
and `keys` in every spelling is jq's. `stdlib/json.arb` runs `names; tally` and
its in-language test (`arb --test stdlib/json.arb`) pins the shape. **The jq leg
of the superset now has ZERO divergences**, machine-checked every run.

**A YAML number keeps its LITERAL, the same way a JSON number does.** It did
not: `ratio: 1.50` printed `1.5` where `yq -o=json` prints `1.50`, while the
JSON reader beside it printed `1.50` for the same text. serde's data model has
nowhere to put a number's source text — a plain scalar reaches the visitor as an
`f64` with the text already discarded — so the YAML reader now composes from the
parser's EVENT stream (`src/yaml.rs`), where a scalar arrives as its raw text,
and builds numbers through the same `num_from_literal` the JSON reader uses. One
number representation, one renderer, both formats.

An INTEGER is the other rule and is not an oversight: it renders as its VALUE,
never its text, because that is what `yq -o=json` prints (`007` -> `7`,
`0xFF` -> `255`, `1_000` -> `1000`). Both rules are probed.

**With that, the jq leg reports ZERO divergences.** So does every other leg: the
whole run is 825 probes, 825 pass, 0 diverged, no allowlist. The script exits
with the divergence count, so a clean run is an exit status and not a summary
line anyone has to read.

**One deviation runs the other way.** For an integer above 2^53, jq's own
arithmetic loses up to an ULP: `jq` answers `true` to
`(-516424571754902561 + 0) == -516424571754902500` when the correctly-rounded
double is `…600`. arb reads every number with Rust's correctly-rounded parser and
prints the shortest decimal that round-trips, so it answers `…600` and differs
from the reference by being right. It is not "fixed" to match, and the generated
sweep in `tests/jqlang.rs` states the tolerance explicitly: the computed path may
differ from jq ONLY above 2^53, and byte-matches everywhere else.

**jq expression bodies are jq VALUES.** A `select(…)` predicate, a `map(…)` body
and a bare arithmetic stage evaluate over JSON values, not over arb's f64
expression language (§6):

* a comparison yields a boolean — `map(. > 1)` is `[false,true,true]`, not `[0,1,1]`;
* `select` uses jq truthiness, where only `false` and `null` are falsy, so
  `select(.a)` keeps `0`, `""`, `[]` and `{}`;
* `==` compares type as well as value (`1` is not `"1"`), and an absent key is
  `null`, so `select(.b == null)` is how "field missing" is spelled;
* `+` is overloaded per type — number add, string concat, array concat, object
  merge, and `null` as the identity on either side — with `-` on arrays (set
  difference), `*` on objects (recursive merge) and on a string (repetition),
  and `/` on two strings (split);
* `%` truncates both operands to integers first, as jq's does. Arb's own
  `map x % 3` keeps §6's f64 remainder; the jq context follows jq, which is the
  same context gating that separates jq `to_entries` from native `entries`;
* nested field paths work inside a body (`select(.a.b > 1)`, `map(.a.b)`), and a
  path may continue THROUGH an iterate (`.users[].name`, `.a[].b.c`);
* a top-level JSON STRING renders RAW, the way `jq -r` prints one: a line reading
  `"hello"` is `hello`, and `"a\"b"` is `a"b`. That holds for the filters that
  emit their input line unchanged — identity `.`, `select(…)` and `values` — as
  well as for the ones that already rendered (`.a`, `.[]`, a slice). Other types
  keep the source line, which is also what jq does with a number it never computed
  (`1.50` stays `1.50`);
* a TYPE mismatch is a hard error, not an answer. `null | .[]`, `3 | .a`,
  `"hello" | .[1]`, `true | length`, `{"a":1} | . + 3`, `.n / 0` and `.n % 0` all
  refuse with `jq: …` on stderr and a non-zero exit, matching what `jq` itself
  does on the same input (exit 5, as jq uses);
* an uncaught error ends its INPUT, not the stream, as jq's main loop does: the
  values that input produced before the error are printed, the error goes to
  stderr, the next line still runs, and the exit status is the LAST line's (5
  when it raised, else 0). A non-string error value is reported as jq words it,
  `jq: (not a string): {"a":1}`. `halt` and `halt_error` end the whole run with
  their own status, and `halt_error` writes its message unprefixed — a string
  raw with no newline, any other value as JSON plus a newline, `null` not at all.
  The TUI, REPL, `serve` and `--test` still show the first error as a refusal;
* `in.json` reads JSON TEXTS, as jq's parser does, not one document per line: a
  pretty-printed document spanning lines is ONE input (rendered compactly), and
  several values on one line (`1 2 3`, `{"a":1}{"a":2}`) are one input each. A
  line holding exactly one value passes through untouched. A line that is no
  part of a document — or an opened document that never balances, or balances
  but does not parse — keeps the text reading below, line by line.
  `input_line_number` counts lines, so a multi-line document reports the line it
  ends on.

Those type rules apply to a line that PARSES as JSON. arb's stream is TEXT and
`jq` has no reading of a non-JSON line at all (it refuses the whole input), so
arb keeps its line-stream behaviour there rather than inventing one: a path
yields `null`, an iterate/slice passes the line through, and an EXPRESSION sees
the line as jq's string — `. * 2` over a line reading `abc` is `abcabc`.

A compare may test strings as well as numbers — `select(.status == "ok")`, and the
ordered forms too (`select(.s < "abd")`), which follow jq's total order
(`null < false < true < numbers < strings < arrays < objects`) — and a key that is
not a bare identifier is reachable through the bracket form, `.["a b"]`. A
subscript keeps its type: `.["0"]` is an object key and `.[0]` is an array index,
so `[1,2] | .["0"]` refuses rather than reading the first element. Path results
follow jq on absent data: an explicit null, a missing key and an out-of-range
index all render as `null`, not as an empty line. (Native `field NAME` is
unchanged — it still yields "" on a miss and still falls back to logfmt, because
it runs over plain-text streams as well as JSON.)

**Where a jq literal and a native verb share a spelling.** arb's pipeline is a
LINE stream, so a jq filter returning one array is emitted as one compact array
line, while the native verb of the same family emits a line per element. The two
spellings are kept distinct wherever they differ:

| spelling | shape | note |
|---|---|---|
| `to_entries` (jq) | one `[{"key":…,"value":…},…]` line | matches jq |
| `entries` (native) | one line per key | SPEC §8, unchanged |
| `keys` (jq, any spelling) | one `["a","b"]` line | matches jq |
| `names` (native) | one line per key | SPEC §8, the renamed native verb |
| `. \| add` (jq) | folds from `null`, so `[]` is `null` | matches jq |
| `add` (native verb) | `[] -> ""`, string-joins a mixed array | SPEC §8, unchanged |
| `. \| length` (jq) | refuses a boolean | matches jq |
| `length` (native verb) | falls back to the raw line's char count | runs over plain text too |

The rule is CONTEXT, not spelling: a body command that begins with a jq literal
(`.`, `select(`, `map(`, `has(`) is handed to the jq front-end whole, and inside
it every builtin answers as jq does. A bare alphanumeric word is looked up in the
native verb table first, and only a word with no native verb falls through to jq.
That gating is what separates jq `to_entries` from native `entries`. A shared
word FOLLOWED by a jq operator (`|`, `,`, `as`, `and`/`or`, arithmetic or a
comparison) is not bare: none of those verbs (`length`, `add`, `first`, `floor`,
`max`, …) takes an argument, so `length | . + 1` and `length as $l | …` are jq
programs. `sort -n` keeps its flags.

**The context rule has one boundary, and `keys` was over it.** The native table is
matched BEFORE the fall-through, so a native verb spelled exactly like a jq
builtin does not merely win the bare word — it SHADOWS jq's builtin, and the
superset contract breaks on that spelling. Every other collision in the table
above is safe because the two spellings DIFFER (`entries` vs `to_entries`, `vals`
vs `values`, `sort_by v` vs `sort_by(.v)`), and jq's own name stays reachable.
`keys` was the single spelling where they did not differ: jq's canonical short
name and arb's native verb were the same word, so no context could route both.

It is resolved by giving the native verb a name of its own. **`names`** is the
line-per-key verb; **`keys` in every spelling is jq's** sorted array. Three
things decide the direction, and they point the same way:

* jq owns the name. `keys` is jq's canonical spelling and has no shorter form,
  while arb's native verbs already prefer the SHORT spelling of a shared idea
  (`vals` beside jq's `values`, `entries` beside jq's `to_entries`). arb had no
  shorter spelling to retreat to, so it took a distinct one instead.
* Nothing is lost. `names` is the same op it always was — line per key, and a
  non-object line passes through untouched, which jq's `keys` refuses to do. Both
  behaviours are now reachable, under names that do not collide.
* The one shipped preset that depended on it moved with it. `stdlib/json.arb`
  runs `names; tally` and its in-language test (`arb --test stdlib/json.arb`)
  pins the line-per-key shape, so the rename is covered by a test that fails if
  it regresses.

With that, `scripts/jq_parity.sh` reports **zero jq divergences**: every probe in
the jq leg byte-matches `jq -rc` on the same input.

### In-language unit tests (`arb --test`)

A spec can carry its own tests: a `test "NAME" { … }` block feeds sample lines
through a query pipeline and asserts the output. `arb --test spec.arb` runs every
block headlessly and exits 0 (all passed) / 1 (any failed), with
[TAP](https://testanything.org/) output — so a dashboard's transforms are
regression-tested in CI, in the same language they're written in.

```
test "keeps 5xx, drops others" {
    given "200 ok" "503 down" "404 x" "500 err"   # input lines (one per arg)
    run { in; match /5\d\d/ }                     # the pipeline (reuses source/out grammar)
    want "503 down" "500 err"                      # expected output lines
}
test "counts errors" {
    given "e" "e" "ok" "e"
    run { in; match /e/; count }                  # a reducer → one scalar line
    want "3"
}
test "jq path" { given "{\"u\":{\"name\":\"bob\"}}"; run { in.json; .u.name }; want "bob" }
```

`run { … }` is the ordinary `source`/`out` body — native verbs and the jq/xpath
literal front-ends are all testable. The output is flattened for comparison: a
scalar renders as one line, `tally`/`count_by` pairs as `key\tvalue`. Test blocks
are ignored by every mode except `--test` (they don't render a widget).

## 9. Widgets ("Tk" register)

```
text .t -label L          tail .t -label L        table .t -cols "a,b,c"
list .t                   gauge .t -label L -max N spark .t
bars .t -label L          histo .t                chart .t
linegauge .t -max N       scatter .t              sparkline .t   # bars / scatter
map .t -res high           calendar .t             # world map / month calendar
logview .t                 heatmap .t              treemap .t     # log / grid / rects
gantt .t                   diff .t                 # time spans / +- coloring
logo .t                    clear .t                rule .t        # splash / spacer / divider
select .s -prompt P -header H     input .i -placeholder P    # interactive
sel .ps                   # per-widget selection list -> .ps.sel (§14)
tabs .t -tabs {a b}       block .t -title T -border  frame .f
.t configure -max 200     # reconfigure (merge opts into a declared widget)
```

Any widget takes `-color NAME` (green/red/yellow/orange/magenta/blue/white/gray,
default cyan) to tint its border and accent — same color in the TUI and web.
`select` is an interactive fuzzy picker (fzf as a one-widget spec; `source`
projects the candidate display, `search` derives a separate match key); `input`
is a live field whose value drives `apply`/`bind`/`out`. The chart family:
`linegauge` (a compact one-line `gauge`), `scatter` (braille scatter of a numeric
series), `sparkline` (a block-bar sparkline, the fixed-height counterpart to
`spark`'s braille line), `map` (a braille world map plotting `lon lat` points from
the stream — geo scatter), and `calendar` (a month grid highlighting days that
appear as `YYYY-MM-DD` in the stream). Scrollable list widgets (`tail`/`list`/
`block`/`frame`) draw a scrollbar when content overflows. `sel` is an in-dashboard
selection list over its own `source` — Up/Down (or a click) move a cursor and its
highlighted row is published as the control value `.<path>.sel`, readable from
`where`/`apply`/`tell`/`send` (the per-widget-named-source selection accessor).

arb wires the full ratatui data-widget set — `Paragraph`, `List`, `Table`,
`Tabs`, `Gauge`, `LineGauge`, `BarChart`, `Sparkline`, `Chart`, `Canvas`
(scatter/map), `Calendar`, `Scrollbar`, `Block` — leaving only the non-data
utilities (`Clear`, `RatatuiLogo`) unused.

## 10. Layout (auto by default)

```
# no grid → widgets auto-tile (vertical flow). Only add geometry to override.
layout horizontal                      # auto-tile in a row instead of a column
grid .a -row 0 -col 0 -span 2          # -span = colspan; -rowspan/-colspan explicit
grid .b -row 1 -col 0

# Proportional tracks (Tk `grid`-style, ratatui Constraints under the hood):
rows "1 2 1"                            # 3 rows; the middle is 2× tall (weights)
cols "20c * 2*"                          # col 0 fixed 20 cells, col 1 weight 1, col 2 weight 2
gap  1                                  # 1 blank cell between every row/column
```

A **track** is `N` (a proportional weight → `Fill`; the common case), `Nc` (a fixed cell count → `Length`), `N%` (a percentage of the
axis → `Percentage`), or `N*` / `*` (a proportional weight → `Fill`; bare `*` is
weight 1). `rows`/`cols` size the grid's tracks (unset = equal weights); a shorter
spec sizes the leading tracks and the rest fill. `gap N` inserts blank cells
between tracks. Without any `grid` cell, widgets auto-tile in the `layout`
direction (`vertical` default, `horizontal` for a row), sized by the flow-axis
track spec when given. The **served web dashboard uses a responsive CSS grid** —
`rows`/`cols`/`gap`/`layout` shape the terminal TUI.

## 11. Binding

```
source .t { in.json; each; where(is5xx); count; every 1s }   # stream → widget via query
.g <- cpu_pct                     # reactive: widget follows a signal
.t <- now() every 1s              # sampled bind
```

## 12. Interactive controls — megafilter/map the passthrough

Controls render AND feed `out`. A control's path used as a value = its current state.

```
filter .q                 # text box  → .q string
facet  .lv -field level   # facets    → .lv selected set
slider .th -field lat -min 0 -max 5s   # → .th value
check  .on -label live     select .k -opts {a b c}

out {
    where(match(.q))            # filter by text box
    where(level in .lv)         # filter by facet selection
    where(lat < .th)            # filter by slider
    map(x => pick(x, {ts msg})) # MAP the passthrough, not just filter
}
```

## 13. Expect — stream reactions

A matching stream line fires an action (space-form args, per §2 — not paren calls):

```
expect /5\d\d/ { alert "5xx"; flash .log red }   # regex → a block of actions
expect /panic|OOM/ beep                          # …or a single action
expect /down/ exec "notify-send arb"
timeout 5s alert "stream idle"           # fire when no new line for 5s (Ns/Nms/Nm)
# actions: set .name V | quit | beep | alert MSG | flash .w COLOR | exec CMD
#          | send "text" (to a spawn -pty child) | { … } (a block, run in order)
```

The block form groups several clauses under one `expect`:

```
expect {
    /panic|OOM/ { alert "crash"; flash .log red }   # each clause: /re/ ACTION
    /5\d\d/     beep
    /shutdown/  quit
}
```

`spawn CMD` (SPEC §7) launches a process whose stdout feeds the stream;
`spawn -pty CMD` runs it on a pseudo-terminal and a `send "text"` action writes
to its stdin, so an `expect { /re/ send "…" }` clause automates it (Expect).

## 14. Events — bind (Tk)

```
bind C-q quit                       # a control key → an action (any §13 action)
bind <Enter> quit                   # Tk named keys: <Enter> <Esc> <Tab> <Key-x>
bind <Click> beep                   # any mouse press → an action
bind <Resize> { alert resized }     # terminal size change → an action
bind C-r { alert reloaded; beep }   # block form
```

**Mouse** (SGR reporting, enabled on the TUI's `/dev/tty`): **left-click** a
control to focus/toggle it (checkbox, facet option), click-drag a `slider`,
click a `tabs` label to select it, click an fzf row to move the cursor
(**double-click** a row to pick it, like Enter). **Right-click** a control
resets it to its empty/default (slider→min, checkbox→off, text/facet→cleared).
**Middle-click** focuses without acting. The **wheel** scrolls: over a
scrollable widget (`tail`, `list`, `text`, `table`, `block`, `frame`) it banks
older rows and walks back toward the live tail; elsewhere it moves the focused
`facet` cursor; in fzf mode it moves the selection. The raw button byte also
carries **Shift/Alt/Ctrl** modifier bits (`mouse_shift`/`mouse_alt`/`mouse_ctrl`).
Everything is decoded from the raw tty byte stream and hit-tested against the
rendered widget rects. To copy text, hold **Shift** and drag for your terminal's
native char-precise selection (arb captures the mouse for its widgets).
**Selection accessor** — a `sel` widget (a per-widget selection list over its own
`source`) publishes its highlighted row as the control value `.<path>.sel`, so a
`ps aux | arb` dashboard with `sel .ps` + `source .ps { in }` exposes `.ps.sel` =
the current process row. Move the cursor with Up/Down (or click a row); the value
updates live and is readable anywhere a control is — `where match(.ps.sel)`,
`apply`, `tell w job(.ps.sel)`, or a `send` to a PTY child. ⬜ OSC-52 whole-widget
copy is deferred (line-granular, terminal-gated — Shift+drag is the better path).

## 15. Actors — Akka-style concurrency

An **actor** is a named behavior with a single scalar `state` and one handler per
message. Handlers run arb expressions (the same fusevm-lowered core as
`map`/`where`/`calc`) over `state`, the message parameters, and any locals they
assign; `reply EXPR` sends a value back to an `ask`/`via` caller.

```
actor worker(state) {
    on job(x) {
        state = state + 1        # per-worker running count
        reply x * x              # value sent back to ask / via
    }
    on reset { state = 0 }
}
```

Handlers are separated like every other verb — a newline or `;` (a `{ … }` block
does **not** end a command, so two handlers on one line is an error).

**Runtime** — one OS thread per actor, each blocking on an `mpsc` mailbox:

| Operation | Meaning |
| --- | --- |
| *spawn* | start an actor with an initial `state`; returns a mailbox ref |
| *send* (tell) | post a message, fire-and-forget |
| *ask* | post a message with a one-shot reply channel, block for `reply` |
| *pool* | a supervised round-robin pool of N copies; a worker whose thread dies is respawned on the next dispatch |

**Pipeline fan-out (`via`)** is the stream-facing consumer — it fans the passthrough
across a pool in parallel, each line's scalar becoming a `job(x)` ask whose reply
is the output line, order preserved:

```
source .out { in; via worker * 8 }   # 8-worker pool
out       { in; via worker }          # default: one worker per hardware thread
```

A pure-map handler (`reply x * x`) is deterministic regardless of pool width. A
handler that mutates `state` is partitioned across workers, so with `N > 1` a line
sees only the state of the worker that handled it (documented non-determinism —
use `via worker * 1` for a single sequential accumulator).

**Session refs + event-driven messaging** — spawn long-lived actors/pools in the
spec and drive them from key/stream/timer events:

```
actor worker(state) { on job(x) { state = state + 1; reply x * x } }

spawn w = worker(0)                # a session actor ref
pool  p = worker * 8              # a supervised session pool
supervise p { on crash { stop } }  # fail-stop (default is restart = respawn)

input .out                                   # a control widget to show a reply
bind C-t tell w job(5)                        # tell: fire-and-forget on a keypress
bind C-a ask .out w job(.th)                  # ask: reply written into `.out`
expect /error/ tell w job(1)                  # drive an actor from a stream match
```

| Form | Meaning |
| --- | --- |
| `spawn NAME = ACTOR(init)` | a single session actor, initial `state` = `init` |
| `pool NAME = ACTOR * N` | a supervised N-worker pool (`* N` optional → one per thread) |
| `supervise NAME { on crash { restart \| stop } }` | crash policy; `restart` (default) respawns a dead worker, `stop` is fail-stop |
| `tell REF MSG(args)` | action: post a message, fire-and-forget |
| `ask .CTRL REF MSG(args)` | action: ask, write the reply into control `.CTRL` |

`args` are arb expressions evaluated when the action fires, with live control
substitution (`.th` → that input's current value). `spawn NAME = …` is
disambiguated from the `spawn CMD` process source by the `=`. The message tell is
`tell` (not `send`) because `send` is the Expect PTY action (§14). Session refs
run only in the interactive TUI; the served web target renders widgets but does
not fire bind/expect actions.

The runtime lives in `src/actor.rs`: declaration + handler compiler + one
`mpsc`-mailbox thread per actor (`spawn`/`send`/`ask`/`pool`, plus `Session` for
the named session refs). `via` is `QueryOp::Via`, evaluated by `actor::run_via`
(rayon fan-out over the pool). `heavy(x)`-style user functions are not part of
arb's expression grammar — handler bodies use arb expressions directly.

## 16. Targets

```
target tui                 # default
target web -port 8080      # served page + WS
theme neon-noir            # one of 31 built-in palettes (arb --list-themes)
theme custom 201 231 93 219 57 53   # a custom 6-index (256-color) palette
set refresh 250            # ms redraw throttle
```

**Themes** — `theme NAME` sets the active color palette from the 31 built-ins
(the storageshower HUD palettes shared with the sibling `iftoprs`/`htoprs` apps:
`neon-sprawl`, `acid-rain`, `neon-noir`, `blade-runner`, `night-city`,
`megacorp`, `zaibatsu`, `iftopcolor`, … — `arb --list-themes` prints them with
swatches). Each is a 6-color palette `(primary, accent, alt, mid, dim, bg)` of
256-color terminal indices; `theme custom c1 c2 c3 c4 c5 c6` supplies your own.
With a theme active the whole TUI recolors from the palette as one system — a
widget with no `-color` takes a slot chosen by its kind (value gauges → accent,
bars → alt, series/plots → mid, text/containers → primary) so a dashboard is
multi-colored like the iftop/htop HUD rather than monochrome, and a themed fzf
picker maps the palette onto fzf's own colour slots (rows → primary, matches →
accent, cursor bar → bg, counter/header/separator → dim). `arb --fzf` starts from
**fzf's palette**, not the arb theme, unless a theme is asked for explicitly
(`--theme NAME`, a spec `theme` directive, or the live `Ctrl-T` chooser) — the
drop-in has to look like `fzf`. An explicit `-color <slot>` (`accent`/`primary`/`alt`/`mid`/`dim`/
`bg`) resolves through the palette too. The fixed semantic names (`-color green`
/`red`/…) remain theme-independent explicit overrides.

**A theme is always active by default** (matching the sibling `iftoprs`/`htoprs`
apps, which default to `neon-sprawl`), so every dashboard — including the stdlib
presets, which set no `-color` — is themed out of the box. Resolution precedence:
`--theme NAME` (per-run) → the spec's own `theme` directive → the `~/.arb/config.toml`
`[ui] theme` global default → the baked `neon-sprawl`. Set the global default with
**`arb --set-theme NAME`** (persists to the config), preview all 31 with
`arb --list-themes`, and opt out to the classic cyan look with `theme off` (or
`--theme off`). At runtime, **`Ctrl-T`** cycles the theme live in every mode
(dashboard, form, and the fzf picker) and saves the choice to `~/.arb` — a
control key, because a bare letter would be swallowed by the megafilter / a text
input. **`Ctrl-G`** toggles a help overlay of the global keys plus the spec's
`bind`s. The served web dashboard keeps zgui-core's own colorscheme picker.

Ships today as `arb --serve --port N`: a std-only HTTP server renders the same
spec as a live browser dashboard, pushing widget data over a WebSocket (hand-rolled
handshake, no dependency) with automatic fallback to polling `/data`. The page is
built from **[`zgui-core`](https://github.com/MenkeTechnologies/zgui-core)** — the
shared cyberpunk web-component toolkit — vendored as a git submodule at
`lib/zgui-core` and bundled into the binary at build time (`build.rs` →
`include_str!`). The page mounts `ZGui.appShell` (splash, filter bar, ⌘K palette,
settings/colorscheme) and renders each widget with the matching component:
`gauge`→`ZGui.gauge`, `chart`→`ZGui.chart`, `spark`→`ZGui.sparkline`,
`bars`/`histo`→`ZGui.statBars`, `table`→`ZGui.dataTable`, containers/log →
`ZGui.card`+`ZGui.logView`, fed live from `/data`. `input` widgets render as
editable fields that `POST /set?name=..&value=..` on change; the server holds a
live input store and re-resolves each widget's pipeline against it every frame,
so a typed field reshapes the browser dashboard just like the TUI megafilter.

## 17. Modules & presets (presets = stdlib script imports)

```
import http                # stdlib or user module by name
import "./mylib.arb"       # file
import gauges as g         # namespaced

# resolution: local → ~/.arb/lib/NAME → installed pkg (~/.arb/pkg/NAME) → stdlib
```

- **Dashboard module (preset):** top-level widget/source/layout stmts. `arb -p http` == `import http`.
- **Component module (library):** exports `fn`s that build widget-groups: `g.cpu(.c)`.

Both are just `.arb` files. Compose: `import http; gauge .mine …; grid .mine -row 2`.

## 18. Package manager (ported from stryke `[ffi.exports]`/`load_cdylib` + znative ABI)

`arb.toml`:
```toml
[package]
name = "arb-k8s"
version = "0.1.0"
license = "MIT"

[deps]
arb-http = "0.2"

[exports]                     # ← stryke.toml [ffi.exports]
modules = ["pods", "nodes"]   # .arb dashboards + components

[exports.native]              # ← znative / load_cdylib
widgets = ["flamegraph"]      # cdylib: new widgets / formats / actors
formats = ["protobuf"]
```

Kinds: **script packages** (pure `.arb`) and **native packages** (`cargo add arb-native`, ship cdylib, stable versioned ABI).

```sh
arb install arb-k8s   arb add arb-http   arb publish   arb search k8s   arb update
```

Distribution: native → crates.io (like fusevm/znative); script → git index + GitHub repos (like the stryke-* family).

**Ships today** (std-only, `git` subprocess as transport — no in-process TLS):
`arb update` clones/pulls the index repo into `~/.arb/registry`; `arb search Q`
greps its `index.json`; `arb install NAME` / `arb add NAME` `git clone`s the
package into `~/.arb/pkg/NAME` and validates its `arb.toml` + entry module before
keeping it (rolled back on failure); `arb uninstall NAME`. A package's `[deps]`
are resolved recursively from the same index, with each dep's version-constraint
**checked** against the index version (`semver`), a visited-set cycle guard,
skip-already-installed, and full rollback of the run if any dep fails or a
constraint is unsatisfiable. The module resolver reads `~/.arb/pkg` as the §17
`pkg` tier, so `import NAME` finds an installed package. **`arb publish
[GIT_URL]`** registers the package for real: it validates it, fast-forward-pulls
the index clone, upserts the package's `{repo, version, desc}` entry into
`index.json`, commits, and pushes to the index remote (default
[`github.com/MenkeTechnologies/arb-registry`](https://github.com/MenkeTechnologies/arb-registry)).
With write access the entry lands directly; when the push fails the commit stays
local and arb prints git's reason with the fork+PR flow -- it never falsely
claims a push succeeded, nor assumes every failure is a denied one.
`GIT_URL` defaults to the package repo's `origin` remote. A package declaring
`[exports.native]` is rejected (native/cdylib loading isn't built — never
installed with an inert native half). Full multi-version semver *resolution*
(one index ref per name today, like the crates.io index tip) and native/cdylib
packages remain future work.

## 19. Ecosystem — "a TUI for every pipeline"

Community publishes `arb-<tool>` packages. `cmd | arb` sniffs the upstream command (or data shape) → resolves the matching package → renders. Every common pipeline (docker/kubectl/psql/nginx/git/systemctl/…) gets a shared, installable dashboard. No registry of shareable pipeline TUIs exists today — this is the ecosystem leg.

**Ships today** — zero-config **data-shape sniffing**: `cmd | arb` (no spec, piped) peeks the first stream lines (via a non-blocking `poll`, so it never delays startup or hangs on an idle producer) and auto-selects the matching stdlib preset — JSON object streams → `json`/`logs`/`nginx`, tool headers → `docker`/`top`/`k8s`, git-log → `git`, CSV/TSV → `table` — replaying the peeked lines so nothing is lost, and falling back to the plain tail on no match. The **upstream-command** leg (identifying the producer process by argv) is deferred: the data-shape leg dominates it cross-platform (macOS pipe-peer matching needs fragile FFI), and covers every motivating producer via its header/shape.

## 20. Architecture (fusevm frontend, original — mechanics ported, semantics fresh)

Deps (lean): `fusevm` (`jit`, `jit-disk-cache`, `aot`, `ffi`), `ratatui`+`crossterm`, `clap`, `regex`, `onig` (Oniguruma, for jq's regex family), `rayon`; the served web dashboard is **std-only** (hand-rolled HTTP + RFC 6455 WebSocket, no async runtime) and renders with the vendored `zgui-core` toolkit (git submodule `lib/zgui-core`, bundled by `build.rs`); REPL: `reedline`+`nu-ansi-term`+`libc`+`toml`; parsers: `serde_json` + `saphyr-parser` (YAML, `src/yaml.rs`) + `toml` + `scraper` (HTML/CSS) + `base64`/`percent-encoding`.

Actual tree:

```
src/lexer.rs     Tcl-flavored reader
src/parser.rs    command + block grammar → AST
src/ast.rs       AST types (Command / Arg)
src/spec.rs      spec interpreter: widgets, source/out pipelines, query-verb
                 parse, import resolution, preset library
src/query.rs     jq/xpath/css/yq engine (pipeline eval over every format)
src/jq.rs        jq-literal front-end: translates `source`/`out` body stages to query ops
src/jqlang.rs    complete jq language engine (parse_json, programs, builtins)
src/jqval.rs     jq value expressions (`select(…)`/`map(…)` bodies, arithmetic stages)
src/jvparse.rs   jq 1.8 JSON text parser port (fallback reader)
src/jsondocs.rs  `in.json`: one item per JSON document
src/xpath.rs     XPath front-end; xpath_syntax.rs (lexer/parser) + xpath_eval.rs (evaluator)
src/yaml.rs      YAML reader (event stream → jq value model); ynode.rs node metadata + writer; yqfmt.rs yq encoders/decoders
src/algo.rs      fzf matching algorithm port; pattern.rs fzf extended-query port; fzf.rs fzf option compatibility
src/sniff.rs     zero-config data-shape sniffing
src/testrun.rs   in-language test runner (`--test`)
src/tiers.rs     fusevm execution-tier probe (`--tiers`)
src/pty.rs       `spawn -pty` pseudo-terminal source
src/rust_ffi.rs  inline `rust { … }` FFI blocks
src/hosted.rs    running arb inside a host process (no `exit()`)
src/err.rs       `SpecError` with source spans
src/actor.rs     actor system (§15): actor/on/reply parse + handler compiler, mpsc-mailbox threads (spawn/send/ask/pool), `via` parallel stream fan-out
src/theme.rs     31 built-in color palettes (storageshower, shared with iftoprs/htoprs) + custom palette; theme-aware color resolution
src/expr.rs      expression layer: fn/lambdas/operators → fusevm::Chunk on the VM
src/stream.rs    stdin ring buffer + stream stats
src/tui.rs       ratatui backend: render, event loop, fzf mode
src/serve.rs     live web server + WebSocket push; renders via zgui-core (appShell + components)
src/web.rs       static HTML snapshot export (--html)
build.rs         bundle lib/zgui-core/webui/*.js + all.css -> one JS/CSS asset, embedded in serve.rs
lib/zgui-core/   git submodule: the shared cyberpunk web-component toolkit (window.ZGui.*)
src/repl.rs      interactive REPL (--repl)
src/pkg.rs       registry client (install/search/update/publish) over a git index
src/lsp.rs       Language Server over stdio (--lsp): diagnostics/symbols/hover/completion/signatureHelp/definition/references/highlight/rename/folding/formatting/semanticTokens
src/dap.rs       Debug Adapter over stdio (--dap): step the stream, regex breakpoints, inspect the paused line/stats/controls
src/cache.rs     rkyv script cache (~/.arb/scripts.rkyv): outer zero-copy rkyv shard, inner bincode AST blob, FxHash+schema key — skips lex+parse for a seen spec
src/banner.rs    startup/help art
src/main.rs      binary entry point; calls `cli::run_argv`
src/cli.rs       CLI (clap) + dispatch + fzf option handling + `--filter`
src/lib.rs       crate root
```

The compute core (expressions, `calc`, `where`) lowers to a `fusevm::Chunk` and
runs on the VM; declarative widget/layout construction is plain Rust and needs no
VM. Language design (lexer/parser/ast/interp/semantics) is arb-original.

A chunk can execute on either of two fusevm tiers — the interpreter, or native
code from the Cranelift block JIT once the chunk has been invoked past
`block_threshold` (default 1). `scripts/expr_paths.sh` runs the expression
corpus through both, pinned via `FUSEVM_JIT_BLOCK_THRESHOLD`, because a
construct the tiers disagree about answers differently for the first row of a
stream than for the rest. One such disagreement is open and is fusevm's, not
arb's: `Op::Div` by a zero divisor yields `Value::Undef` (prints `0`) on the
interpreter and IEEE infinity in compiled code, so `map x / 0` over a stream
prints `0` and then `1.7976931348623157e+308` (the infinity, clamped on output
as above), and `where x / 0 > 1` keeps a different subset of identical lines. arb
does not paper over it locally — the op is shared with every sibling frontend.
The clamp does not hide it either: `Value::Undef` never reaches the non-finite
branch, so clamping moves only the compiled side and the two tiers still differ.

All SPEC modules now have code (script-package registry + the actor system
included; native/cdylib packages remain future work).

## 21. Milestones

Status: ✅ shipped · 🟡 partial · ⬜ planned · ❌ out of scope.

0. ✅ **Walking skeleton** — `echo hi | arb -e 'text .t <- in'`: lex→parse→lower→fusevm→one ratatui widget from stdin.
1. ✅ Core widgets + auto-layout + `source`/query basics.
2. ✅ Presets/imports + stdlib (logs/http/json/table/top/metrics) + module namespacing `import X as Y` (prefixes widget paths, `apply`, control refs, `set`/`flash` targets).
3. ✅ Interactive controls + `out` passthrough shaping (megafilter/map): `input`/`apply`, the `filter`/`facet`/`slider`/`check` control widgets (interactive in both the TUI and the served web dashboard, incl. dynamic `-field` facet candidates), and control-path predicates — numeric `where lat < .th`, string `where match(.q)`, and set `where level in .lv`.
4. ✅ Expect reactions + events/bind — `expect /re/ ACTION` and the multi-clause `expect { /re/ ACTION; … }` block, `bind C-<key> ACTION` with actions `set`/`quit`/`beep`/`alert`/`flash`/`exec` and `{ … }` block form; Tk named keys `<Enter>`/`<Esc>`/`<Tab>`/`<Key-x>`; `timeout Ns ACTION` idle reactions; `spawn CMD` process input source, `spawn -pty CMD` + the `send "text"` action (Expect-style automation of a PTY child). The `sel` selection widget publishes a widget's highlighted row as `.<path>.sel` for `where`/`apply`/`tell`/`send` to consume (§9).
5. ✅ Web target — `arb --serve` HTTP + WebSocket live dashboard rendered with the `zgui-core` component toolkit (appShell + per-widget components); `arb --html` static export.
6. ✅ Actors — Akka-style message-passing (§15): `actor NAME(state) { on MSG(p) { … reply EXPR } }` declarations compiled to `ActorDef`; a real runtime of one `mpsc`-mailbox OS thread per actor with *spawn* / *send* (tell) / *ask* (await reply) / supervised round-robin *pool* (respawns a dead worker); handler bodies run arb expressions (fusevm) over `state` + params + locals. Two surfaces: the `via NAME * N` pipeline op fans the stream across a pool in parallel (rayon), order preserved; and the session-ref surface — top-level `spawn NAME = ACTOR(init)` / `pool NAME = ACTOR * N` bindings, a `supervise NAME { on crash { restart \| stop } }` crash policy, and the `tell REF MSG(args)` / `ask .CTRL REF MSG(args)` bind/expect actions that drive them (interactive TUI).
7. ✅ Package manager — local preset library (`--save`/`--install`/`--uninstall`/`--installed`) + a networked registry over a git index hosted on GitHub (`arb update`/`search`/`install`/`add`/`uninstall`/`publish`, `~/.arb/pkg` resolver tier, transitive `[deps]` with semver constraint-checking). `arb publish` upserts the package's entry into the index and pushes it (default registry `github.com/MenkeTechnologies/arb-registry`). *(native/cdylib packages + multi-version semver resolution: ⬜)*
8. ✅ LSP/DAP — `arb --lsp` ships a full server: diagnostics (real source ranges, UTF-16 columns), `documentSymbol`, `hover`, `completion` (CORPUS verbs + dot-context `.path` names + widget `-flags`), `signatureHelp`, `definition`/`references`/`documentHighlight`/`rename` over widget `.path` names, `foldingRange`, `formatting`, and `semanticTokens/full`. `arb --dap` is a real steppable debugger over the stream model: each incoming line is a step, breakpoints are regex predicates (a `SourceBreakpoint.condition`, or unconditional = single-step), function breakpoints (`setFunctionBreakpoints`) name a query VERB and stop when the paused line reaches that stage — a name arb has no verb for comes back `verified: false` rather than being silently accepted — the stack trace is the query-pipeline stages, scopes expose the matched line + stream stats + control values, and `evaluate` runs arb's real expression evaluator against the paused line. The `program` (spec) and `input` (data file) come from the `launch` request since DAP owns stdio; `stepIn`/`stepOut` collapse to `next` (a stream has no call nesting). Diagnostics anchor to the offending verb even when nested inside a `source`/`out` body (not the enclosing directive). *(per-token argument precision — squiggle the `/re/` itself, not its verb — ⬜)*