# Performance & Optimization Decisions
This document is the decision log for every optimization in Weir. Each entry states the problem, the chosen solution, the rejected alternatives, and the verification method.
---
## Decision log
### 1. GPU telemetry (popcount + reduce) at 64-word threshold
**Problem:** Fixed-point telemetry needs to count set bits and changed bits per iteration. On the host this is a linear scan; on the GPU it requires two dispatches (`bitset_popcount` then `reduce_sum`) plus a scalar download.
**Decision:** Gate GPU telemetry behind the `gpu-telemetry` feature. Use it only when the frontier exceeds **64 words** (2,048 bits). Below that threshold, host-side `count_ones()` is faster because dispatch overhead dominates.
**Rejected alternatives:**
- Always-GPU telemetry: regresses small-graph throughput by 2–5× in micro-benchmarks.
- Always-host telemetry: leaves large-domain profiling data on the table.
**Verification:**
- `benches/fixed_point_scratch_telemetry.rs` measures host vs GPU at 1k / 10k / 100k / 1M bits.
- Unit test in `src/fixed_point_scratch/tests.rs` asserts that the 31-word path falls back to host count.
---
### 2. `bitset::equal` for convergence instead of `memcmp`
**Problem:** The fixed-point loop needs a convergence check every iteration. A naive host `memcmp` is fast but requires a full byte-vector comparison and cannot be expressed as a Vyre program for GPU-side checks.
**Decision:** Use `vyre_primitives::bitset::equal` (word-wise equality with scalar output) as the convergence primitive. On the host it compiles to the same LLVM pattern as `memcmp` for power-of-two lengths. On the GPU it becomes a single dispatch that returns a scalar `1` or `0`, unifying host and device convergence paths.
**Rejected alternatives:**
- Raw `memcmp` on host + custom GPU kernel on device: divergent code paths, harder to parity-test.
- Per-word loop on host: slower for large bitsets; LLVM does not always vectorize it.
**Verification:**
- `benches/fixed_point_scratch_telemetry.rs` includes a convergence-check benchmark comparing host slice-equality vs GPU dispatch.
- Every `df_*` closure test exercises at least one convergence iteration.
---
### 3. `ScratchPool` for zero-allocation staging
**Problem:** Fixed-point iterations allocate temporary vectors (frontiers, outputs, tail buffers) on every loop. In a 100-iteration solve this becomes thousands of allocations.
**Decision:** Introduce `ScratchPool<T>`: a `Vec<T>` wrapper that `clear()`s logical contents while preserving capacity. It implements `Deref<Target = Vec<T>>` so existing code pushes and indexes without boilerplate. A single `FixedPointScratch` struct owns one pool per buffer type and is passed by `&mut` into every iteration.
**Rejected alternatives:**
- `bumpalo` arena: adds `unsafe` and lifetime complexity; Weir's scratch lifetime is simpler than a general arena.
- `smallvec` inline buffers: frontiers often exceed inline capacity; the win is marginal and the code is noisier.
**Verification:**
- `tests/fixed_point_scratch_contract.rs` asserts that warm scratch does not reallocate.
- `benches/resident_fixed_point_hot_path.rs` measures cold-vs-warm allocation counts.
- IFDS resident tests assert "reusing single-query scratch must not allocate transient seed staging buffers."
---
### 4. SIMD oracle is feature-gated and performance-neutral
**Problem:** The CPU parity oracle for bitset closure (`csr_bitset_closure`) is on the critical path of fuzzing and property testing. SIMD (AVX2 on x86_64) could accelerate it, but we do not want to introduce platform-specific code into the default build.
**Decision:** Gate the SIMD path behind the `simd-oracle` feature. When enabled **and** the host is `x86_64`, forward and reverse propagation use `simd::propagate_forward_fast` / `propagate_reverse_fast`. When disabled or on non-x86_64, the stable scalar path is used.
**Performance policy:** The SIMD path must be **performance-neutral or better** on x86_64; it must never regress the scalar path on the same hardware. The scalar path remains the reference of truth for parity.
**Rejected alternatives:**
- Default-enable SIMD: breaks builds on non-x86_64 targets and complicates cross-compilation.
- `target-cpu=native` conditional compilation: too aggressive for a library crate; consumers may have different CPUs.
**Verification:**
- `benches/bitset_closure_oracle.rs` compares scalar vs SIMD at four graph sizes.
- `tests/df_fuzz_bitset_oracles.rs` and `tests/df_property_bitset_oracles.rs` run both paths (when enabled) against the same random inputs.
---
### 5. Dominator GPU threshold at 1,000 nodes
**Problem:** SSA dominator construction has both a CPU iterative path (Cooper–Harvey–Kennedy) and a GPU lane-0 kernel path. The GPU path has fixed overhead (buffer upload, kernel launch, scalar download) that is only amortized on large CFGs.
**Decision:** Route CFGs above **`DOMINATOR_GPU_THRESHOLD = 1_000`** blocks to the GPU path. At or below 1,000 blocks, the CPU path is strictly faster. If the GPU path fails for any reason (no backend, compile error), fall back to CPU transparently.
**Rejected alternatives:**
- Always-GPU: regresses small-function compilation throughput by an order of magnitude.
- Dynamic threshold based on backend probe latency: adds nondeterminism and complicates reproducible benchmarks.
**Verification:**
- `tests/test_ssa.rs` contains `dominator_threshold_tests`:
- `threshold_exactly_1000_uses_cpu`
- `threshold_1001_attempts_gpu`
- `cpu_and_gpu_paths_agree_on_linear_chain`
- `benches/dominators_cpu_oracle.rs` scales the CPU oracle up to 100k nodes to confirm linear scaling.
---
## Benchmark methodology
All benchmarks use **Criterion** with the following conventions:
1. **Black-box inputs:** Every benchmark wraps the subject with `criterion::black_box` to prevent the compiler from constant-folding fixture data.
2. **Throughput annotation:** Benchmarks declare `Throughput::Elements(n)` so Criterion reports ns/element as well as ns/iter.
3. **Warm-up:** Resident benchmarks perform at least one "warm" iteration before measurement to ensure scratch pools and resident buffers are allocated.
4. **Stable fixtures:** Micro-benchmarks use fixed fixtures (4-node diamond, linear chain, binary tree) so results are comparable across commits. Scale benchmarks sweep 1k → 10k → 100k → 1M.
### Running benchmarks
```sh
# Micro-LO regression gates
cargo bench --bench primitive_micro
# Scale-axis ladder
cargo bench --bench primitive_scale
# Resident hot path (cold vs warm upload)
cargo bench --bench resident_fixed_point_hot_path
# IFDS resident hot path (requires cpu-parity)
cargo bench --bench ifds_direct_resident_hot_path --features cpu-parity
# Telemetry host vs GPU (requires gpu-telemetry)
cargo bench --bench fixed_point_scratch_telemetry --features gpu-telemetry
# Dominator oracle scaling
cargo bench --bench dominators_cpu_oracle --features cpu-parity
# Bitset closure SIMD parity (requires simd-oracle + x86_64)
cargo bench --bench bitset_closure_oracle --features cpu-parity
```
---
## Performance budget policy
| `stable_csr_layout_hash` | 4-node diamond CSR, 4 edges | 1,200 | 1.15× |
| `fold_summary_partials` | `DEFAULT_PARTIAL_BINS` triples (3,072 words) | 1,800 | 1.20× |
| `try_compute_dominators` | 4-block diamond CFG | 8,500 | 1.25× |
| `resident_fixed_point_hot_cached_graph` | 2-node reaching plan, window=2 | see `resident_fixed_point_hot_path` bench | 1.20× vs local baseline file |
**Rules:**
1. A regression gate failure is **not** automatically a bug; it is a signal to write a decision entry explaining why the change is acceptable or to revert.
2. New primitives must include a baseline and regression factor before they can merge to `main`.
3. Baselines are calibrated on the release-train host (`cargo bench -p weir --bench primitive_micro`). CI compares against the constants in `benches/primitive_micro.rs`.
---
## How to profile Weir
### Criterion reports
After running any bench, open the HTML report:
```sh
target/criterion/report/index.html
```
### `perf` on Linux
```sh
# Build the benchmark binary first
cargo bench --bench primitive_micro -- --noplot
# Find the binary path and profile it
perf record -g target/release/deps/primitive_micro-* --bench
perf report
```
### Cache-grind / Call-grind
```sh
valgrind --tool=callgrind target/release/deps/primitive_micro-* --bench
kcachegrind callgrind.out.*
```
### GPU-side profiling
Weir does not emit raw CUDA or WGSL; GPU work is dispatched through Vyre. Use the backend-specific profiling hooks:
- **CUDA:** `vyre-driver-cuda` exposes dispatch-timing fields in `TimedDispatchResult`.
- **WGPU:** `vyre-driver-wgpu` (when enabled) integrates with `wgpu::ComputePassTimestampWrites`.
The `fixed_point_scratch_telemetry` benchmarks demonstrate decoding `TimedDispatchResult::device_ns` for kernel-level timing.
---
## Hot-path rules enforced in code
- No `format!()` on GPU decode success paths; cold diagnostics use `error_format` + `write!`.
- Dense `u32` domains use `dense_domain::DenseU32Slots` instead of `HashMap` where ids are bounded.
- `dispatch_decode::unpack_exact_u32_into` reuses caller `Vec<u32>` and uses `bytemuck` on little-endian hosts.
- CSR layout hashing uses word-slice FNV mixing without per-word `to_le_bytes` allocation on LE hosts.