1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
;;; tensor_bench.lisp — the Phase 3.1/3.3 tensor training benchmark.
;;; Timings, not golden output — do NOT add to run_tests.sh.
;;;
;;; Re-records the numbers ROADMAP 3.1 and 3.3 quote, all from ONE machine state:
;;; 8x16->8 x1000 SGD steps — interpreted graph-grad vs compiled graph-compile-grad
;;; 64x256->64 x100 SGD steps — same two paths
;;; Run benchmarks/tensor_torch_bench.py for the 1-thread float64 PyTorch side;
;;; it uses the same shapes, step counts, LR and inits, and the two sides should
;;; land on the SAME final loss (that agreement is what proves the loops match).
;;;
;;; Why absolutes drift: these are wall-clock on whatever machine/thermal state
;;; you run them on. The RATIO is the durable claim — see ROADMAP 3.1.
;;;
;;; JIT marshalling, cut in two passes. Each A/B'd on one machine state with
;;; this file, medians of 3; every step verified bit-identical to graph-grad
;;; across 5 shapes / 37,353 values.
;;;
;;; v0.43.0 — the generated kernel stopped copying: it borrows its input tensors
;;; (was `.to_vec()` per input, ~295 KB/call at 64x256->64) and borrows on the
;;; SumTo same-shape identity (was a clone, ~164 KB/call).
;;; 64x256->64 x100 JIT 143.3 ms -> 130.6 ms (-8.9%)
;;; 8x16->8 x1000 JIT 6.18 ms -> 5.78 ms (-6.5%)
;;; Narrowed the gap to the interpreter but did NOT flip it (~5% slower still).
;;;
;;; v0.44.0 — per-argument-pointer ABI: the caller stopped copying too (was one
;;; flat buffer in + one out, ~885 KB/call at 64x256->64).
;;; 64x256->64 x100 JIT 127.5 ms -> 114.6 ms (-10.1%)
;;; 8x16->8 x1000 JIT ~5.78 ms -> ~5.75 ms (flat — little to marshal)
;;; THIS FLIPPED IT: at 64x256->64 the JIT now BEATS graph-grad (~1.14x), where
;;; it had been ~1.12x slower. Cumulative 143.3 -> 114.6 ms (-20%).
;;;
;;; graph-grad (interpreted) is untouched by both — the medium interpreted row
;;; is noisy (~117-134 ms run to run), so compare JIT numbers, which are stable.
;;;
;;; v0.50.0 — AVX2 without touching a single accumulation: the JIT rustc gets
;;; -C target-feature=+avx,+avx2 when the running CPU has AVX2 (folded into
;;; the cache hash; never target-cpu=native, so the flag says exactly what
;;; the .so contains), and the interpreter's two matmuls share one
;;; runtime-dispatched kernel (graph_ir::matmul_ikj, #[target_feature] AVX2
;;; version of the same body). Rust never FMA-contracts or reassociates, so
;;; wider vectors change speed only — proven by both final-loss values below
;;; landing bit-identical to v0.44, through 100/1000 SGD steps on both paths.
;;; 64x256->64 x100 JIT ~110.8 ms -> ~98.7 ms (-10.9%, medians of 3 runs)
;;; 8x16->8 x1000 JIT ~5.69 ms -> ~5.47 ms (-3.9%)
;;; Cumulative JIT medium: 143.3 -> ~98.7 ms (-31%).
;;; MEASURED CEILING NOTE (Intel N95): Gracemont E-cores double-pump 256-bit
;;; ops, so AVX2 alone is worth only 1.07-1.24x on the matmul kernel here;
;;; the rest of PyTorch's remaining lead is FMA, which fuses rounding and is
;;; BANNED by the bit-for-bit claim. Blocked/tiled matmul at these shapes
;;; was REFUTED as a ~2x lane on this machine (kernel already at ~67% of
;;; non-FMA peak). Next bit-preserving lever: threading rows.
;;;
;;; v0.51.0 — row-parallel interpreter matmul: graph_ir::matmul_ikj splits
;;; output rows across a persistent fixed-assignment pool (worker w owns
;;; band w+1, dispatcher owns band 0; no work-stealing, so no cross-job
;;; straggler hazard) above the measured ~64k mul-add crossover;
;;; RUSTY_MM_THREADS overrides (<=1 = serial). Every band runs the same
;;; kernel body -> per-element op order untouched; verified bit-identical
;;; serial-vs-pooled via save-model diff on 6 shapes (incl. 67 rows,
;;; non-divisible) AND both final losses below unchanged.
;;; 64x256->64 x100 graph-grad (interp) ~106 -> ~88 ms median of 3 runs
;;; (-17%; best run 76 ms; spread widened 76-96 — 4 shared E-cores)
;;; 8x16->8 x1000 unchanged ~33 ms (below crossover, stays serial)
;;; JIT rows unchanged (~99 ms medium) — the FUSED KERNEL IS STILL
;;; SINGLE-THREADED, so the interpreter re-took the medium lead. Pool
;;; micro-bench: 2.07-2.25x at the three training shapes, ~2.3x plateau
;;; at 512^3 (4 E-cores, shared L2). Threading the JIT matmul codegen is
;;; the open follow-up.
;;; PyTorch note: tensor_torch_bench.py pins 1 thread BY DESIGN (kernel
;;; quality comparison). Report threaded-Rusty numbers as their own row,
;;; never against the 1-thread torch row.
;;;
;;; v0.52.0 — fused grad kernels reach the same pool: the NativeGrad ABI grew
;;; a third parameter (graph_ir::mm_bridge) and codegen emits a bridge call
;;; for matmuls over the crossover (static decision — shapes are
;;; compile-time constants), keeping inline loops under it. Bits inherited
;;; from v0.51.0, and re-proven: interp-vs-JIT exact equality on 4 shapes
;;; (incl. relu-boundary: zeroed input row + zero bias), serial-vs-pooled
;;; save-model diff byte-identical (5.9 MB), both final losses unchanged.
;;; 64x256->64 x100 JIT ~98.7 -> ~76 ms median of 3 runs (-23%;
;;; cumulative 143.3 -> ~76, -47%). Row now shares the pool's
;;; run-to-run spread (70.6-78.2) — quote median with spread.
;;; 8x16->8 x1000 unchanged ~5.7 ms (below crossover, inline path).
;;; JIT and interpreter are now neck-and-neck at medium (both pooled;
;;; remaining difference is inside the noise).
;;;
;;; Loss = mean((relu(xW+b) - t)^2). The graph IR has no capture, so everything
;;; the loss needs is a param or a literal; the mean divisor is passed as `nn`.
;;; Inputs are integer-derived /8 — exactly representable, so Rusty and torch
;;; start from bit-identical data.
;; One SGD run. `g` is whatever computes (loss gx gW gb gnn) — graph-grad itself
;; (interpreted: rebuilds+optimizes the graph every call) or a compiled
;; NativeGrad from graph-compile-grad. Identical loop either way.
;; median of 3 without a general sort: total minus the extremes
;; median of REPS timed runs, so a single thermal blip can't set the record