rscrypto 0.6.2

# BLAKE3 ASM Ownership Notebook

Started: 2026-06-24

Goal: replace the upstream-derived BLAKE3 x86_64 assembly with rscrypto-owned
assembly that is smaller, easier to audit, more thoroughly tested, and faster
on the rows where the official `blake3` crate is still ahead.

This is an ownership project, not a cleanup project. Upstream-derived files stay
only when measurements prove they still earn their place.

## Phase 1 Closeout

Completed: 2026-06-25

Phase 1 definition: replace vendored/upstream-derived BLAKE3 ASM where an
rscrypto-owned implementation is faster or neutral, remove dead vendored
surface area, and make provenance explicit for every BLAKE3 assembly file.

Status: complete for this phase. BLAKE3 is not fully rscrypto-owned yet, but the
current production routing is the measured best mix from this pass.

Retained changes:

- Deleted the upstream-derived SSE4.1 x86 assembly files and routed SSE4.1
  compression through owned Rust intrinsics.
- Removed unused x86 XOF assembly exports that no longer had Rust callsites.
- Removed live AVX2 `compress_in_place` assembly and routed callers through the
  owned AVX2 intrinsic compressor.
- Routed AVX2 full contiguous `hash_many` batches through owned intrinsics.
- Promoted measured AVX2 chunk tails `1/2/3/5/6/7` and parent reductions
  `1/2/3/5/6/7/8` to owned paths.
- Promoted AVX-512 15-chunk and 15-parent tails to owned paths behind the
  AVX512DQ guard.
- Added diagnostic BLAKE3 kernel selectors and benchmark rows for owned versus
  assembly paths.
- Updated BLAKE3 assembly provenance headers and kept the assembly ledger green.

Retained upstream-derived assembly because it is still faster:

- AVX2 4-lane chunk tail and 4-parent tail paths.
- AVX-512 full `hash_many` batches and most AVX-512 sub-degree tails.
- AVX-512 `compress_in_place`.
- AVX-512 `xof_many`.

Rejected candidates:

| Candidate | Measured result | Decision |
| --- | --- | --- |
| AVX-512 full owned `hash_many` | owned ~2.65 us vs asm ~2.41 us for 16 chunks on Sapphire Rapids | keep asm |
| AVX-512 owned compress | owned ~978.6 ns vs asm ~889.8 ns for 1024B one-chunk digest on Sapphire Rapids | keep asm |
| AVX-512 owned XOF-output cascade | regressed 4096B output by about 4.65% | keep `xof_many` asm |
| AVX2 exact-four straight-line chain | byte-correct but slower on 256B digest/keyed rows | keep current route |

Current provenance state:

- BLAKE3 x86_64 AVX2/AVX-512 assembly files are still external-derived from
  upstream BLAKE3 1.8.5 and carry source-path/SPDX headers.
- BLAKE3 AArch64 one-chunk assembly files are marked `rscrypto-owned`.
- `scripts/check/asm-ledger.sh` reports `68 assembly files, 13 owned, 55
  external-derived, 0 candidates`.

Verification run for the phase-one closeout:

```bash
cargo check --features 'blake3,diag,parallel' --bench blake3
cargo test --features 'blake3,diag,parallel' hashes::crypto::blake3::tests::diag_
git diff --check
scripts/check/asm-ledger.sh
mutagen sync flush rscrypto-linux
ssh ubuntu@linux-dev 'source ~/.cargo/env && cd /home/ubuntu/rscrypto && cargo check --features "blake3,diag,parallel" --bench blake3'
ssh ubuntu@linux-dev 'source ~/.cargo/env && cd /home/ubuntu/rscrypto && cargo test --features "blake3,diag,parallel" blake3'
```

No `ct.toml` update is needed for phase 1. The retained production branches are
selected by public input length and CPU capability and preserve the existing
BLAKE3 compression/keyed semantics. Update `ct.toml` only if a future owned
entrypoint changes the keyed/secret boundary or bypasses an existing
constant-time harness.

## Current Baseline

Source benchmark run:
`benchmark_results/2026-06-22/linux`, commit
`b978c2ca45611325850d7f1af94718e497acde50`.

Generated with:

```bash
python3 scripts/bench/blake3-gap-report.py --root benchmark_results/2026-06-22/linux --top 18
```

Ratio is `official blake3 time / rscrypto time`; higher means rscrypto is
faster. Wins are `>1.05x`, ties are `0.95x..1.05x`, losses are `<0.95x`.

### Overall

| Scope | Rows | W/T/L | Geomean | Median |
| --- | ---: | ---: | ---: | ---: |
| All parsed BLAKE3 rows | 432 | 233/166/33 | 1.412x | 1.082x |
| x86_64 rows | 192 | 87/88/17 | 1.271x | 1.036x |
| AArch64 rows | 96 | 44/46/6 | 1.437x | 1.048x |

### x86_64 By Platform

| Scope | Rows | W/T/L | Geomean | Median |
| --- | ---: | ---: | ---: | ---: |
| AMD Zen4 | 48 | 25/23/0 | 1.365x | 1.053x |
| AMD Zen5 | 48 | 20/21/7 | 1.327x | 1.030x |
| Intel Ice Lake | 48 | 17/27/4 | 1.164x | 1.008x |
| Intel Sapphire Rapids | 48 | 25/17/6 | 1.239x | 1.077x |

### x86_64 By Operation

| Scope | Rows | W/T/L | Geomean | Median |
| --- | ---: | ---: | ---: | ---: |
| oneshot | 44 | 19/21/4 | 1.204x | 1.042x |
| keyed | 44 | 21/21/2 | 1.237x | 1.036x |
| derive-key | 44 | 35/9/0 | 1.702x | 1.753x |
| streaming | 16 | 1/11/4 | 0.971x | 0.991x |
| xof | 44 | 11/26/7 | 1.136x | 0.999x |

### Worst x86_64 Rows

| Platform | Op | Size | Ratio | Needed Reduction | rscrypto | official blake3 |
| --- | --- | ---: | ---: | ---: | ---: | ---: |
| AMD Zen5 | `streaming` | 64B | 0.822x | 21.7% | 1.73 ms | 1.43 ms |
| Intel Ice Lake | `keyed` | 256 | 0.829x | 20.6% | 255.57 ns | 211.86 ns |
| AMD Zen5 | `keyed` | 256 | 0.853x | 17.2% | 347.60 ns | 296.51 ns |
| Intel Sapphire Rapids | `streaming` | 65536B | 0.872x | 14.6% | 191.38 us | 166.94 us |
| Intel Ice Lake | `streaming` | 64B | 0.874x | 14.4% | 1.12 ms | 977.32 us |
| Intel Sapphire Rapids | `xof` | 0 | 0.875x | 14.3% | 58.85 ns | 51.48 ns |
| Intel Ice Lake | `oneshot` | 256 | 0.902x | 10.9% | 255.39 ns | 230.37 ns |
| Intel Sapphire Rapids | `streaming` | 64B | 0.904x | 10.7% | 1.20 ms | 1.09 ms |
| Intel Sapphire Rapids | `xof` | 64 | 0.920x | 8.7% | 66.25 ns | 60.94 ns |
| AMD Zen5 | `xof` | 64 | 0.925x | 8.1% | 97.35 ns | 90.03 ns |
| Intel Sapphire Rapids | `xof` | 32 | 0.927x | 7.8% | 58.85 ns | 54.58 ns |
| AMD Zen5 | `oneshot` | 256 | 0.931x | 7.4% | 348.00 ns | 324.00 ns |
| AMD Zen5 | `xof` | 1 | 0.934x | 7.1% | 101.62 ns | 94.88 ns |
| AMD Zen5 | `xof` | 32 | 0.935x | 6.9% | 101.53 ns | 94.94 ns |
| Intel Ice Lake | `xof` | 0 | 0.936x | 6.8% | 56.58 ns | 52.98 ns |
| AMD Zen5 | `oneshot` | 0 | 0.938x | 6.6% | 89.88 ns | 84.31 ns |
| Intel Sapphire Rapids | `oneshot` | 65536 | 0.949x | 5.3% | 10.92 us | 10.37 us |
| AMD Zen4 | `xof` | 0 | 0.956x | 4.7% | 97.92 ns | 93.56 ns |

## 2026-06-24 Diagnostic Slice

Added a `diag`-only forced-kernel surface for BLAKE3 so benchmark rows can be
split by backend instead of inferred from dispatch:

- `diag_blake3_digest_with_kernel`
- `diag_blake3_keyed_digest_with_kernel`
- `diag_blake3_xof_with_kernel`
- `diag_blake3_streaming_digest_with_kernel`

The bench now emits `rscrypto-{kernel}` rows under `--features diag`, plus the
runtime/static capability printout. The normal public `rscrypto` rows are
unchanged, so the gap report remains stable.

Verification:

```bash
cargo check --features 'blake3,diag,parallel' --bench blake3
cargo test --features 'blake3,diag,parallel' hashes::crypto::blake3::tests::diag_forced_kernels_match_normal_paths
```

### macOS AArch64

Host: Apple AArch64, NEON available.

Main result: forcing NEON is a bad tiny-input policy on this host. For keyed
64/256/1024 byte inputs, forced NEON was slower than the current public path.
The current dispatch already mostly does the right thing for tiny keyed and
oneshot rows. Large rows are healthy: keyed 1 MiB was about 274 us for rscrypto
versus about 602 us for upstream.

Streaming 64B remains worth revisiting on AArch64. Public rscrypto was about
1.427 ms, forced portable about 1.395 ms, and upstream about 1.374 ms. That is
not an x86 ASM problem; it is update/finalize policy or state overhead.

### Linux x86_64

Host: `linux-dev`, Intel Sapphire Rapids class, AVX-512 and AMX available.

Forced keyed rows showed the current public path is already using the right
high-tier x86 kernel for small and medium inputs. On this host, keyed 256B was
about 221 ns for rscrypto versus about 231 ns for upstream; keyed 1 MiB was
about 110 us for rscrypto versus about 163 us for upstream. Do not spend the
next iteration on keyed dispatch for Sapphire Rapids.

The useful gaps were streaming and short XOF:

| Row | Before | After | Upstream | Result |
| --- | ---: | ---: | ---: | --- |
| `streaming/64B` | 1.200 ms | 1.010 ms | 1.067 ms | ~16% faster, now ahead |
| `streaming/4096B` | 286.5 us | 292.1 us | 290.3 us | small regression; needs retune |
| `streaming/16384B` | 181.4 us | 177.6 us | 178.2 us | ~2% faster, now ahead |
| `streaming/65536B` | 165.8 us | 164.8 us | 165.3 us | flat-to-better |
| `xof/0` | 64.6 ns | 62.0 ns | 51.2 ns | ~3% faster, still behind |
| `xof/32` | 65.0 ns | 61.2 ns | 55.6 ns | ~5% faster, still behind |
| `xof/64` | 72.2 ns | 69.2 ns | 60.0 ns | ~3% faster, still behind |
| `xof/4096` | 1.091 us | 1.056 us | 1.076 us | ~3% faster, now ahead |
| `xof/65536` | 11.43 us | 10.41 us | 10.58 us | noisy but favorable |

Changes retained from this slice:

- AVX-512 x86 profiles now use AVX-512 for the streaming per-block kernel
  instead of AVX2. This closes the `streaming/64B` gap on Sapphire Rapids and
  improves 16 KiB streaming, but the 4 KiB row needs a more precise policy.
- One-block root output for AVX-512 now decodes block bytes and uses the
  AVX-512 root-output block primitive instead of routing through the SSE4.1
  byte helper.

### Linux x86_64 Follow-up: Streaming Retune Rejected

Tried a size-aware streaming policy that kept the AVX-512 public profile but
sent 2..7 chunk updates through AVX2. This was not retained.

| Row | AVX-512 stream policy | Size-aware experiment | Result |
| --- | ---: | ---: | --- |
| `streaming/64B` | ~1.010 ms | ~1.006 ms | preserved |
| `streaming/4096B` | ~292.1 us | ~293.4 us | no recovery |
| `streaming/16384B` | ~177.6 us | ~180.6 us | regressed |
| `streaming/65536B` | ~164.8 us | ~170.3 us | regressed |

Conclusion: a coarse update-size switch is ornamental complexity. The 4 KiB
streaming row needs a real root-cause pass through update/finalize state costs
and chunk flush behavior, not another dispatch threshold.

### Linux x86_64 Follow-up: AVX-512 XOF-many Retained

Routed AVX-512 full-block root XOF output through the existing
`rscrypto_blake3_xof_many_avx512` assembly entrypoint for any number of full
64-byte output blocks. This is a dispatch/output-path win, not ASM ownership
yet, because it still calls the external-derived AVX-512 symbol.

Command:

```bash
cargo bench --features 'blake3,parallel,diag' --bench blake3 -- blake3/xof --sample-size 10 --warm-up-time 1 --measurement-time 1 --noplot
```

Host: `linux-dev`, Intel Sapphire Rapids class.

| Row | Previous rscrypto | Current rscrypto | Current upstream | Result |
| --- | ---: | ---: | ---: | --- |
| `xof/0` | ~62.0 ns | 51.4 ns | 51.0 ns | closed; tie |
| `xof/1` | ~84.8 ns | 78.3 ns | 80.9 ns | now ahead |
| `xof/32` | ~61.2 ns | 53.0 ns | 56.4 ns | now ahead |
| `xof/64` | ~69.2 ns | 58.3 ns | 55.3 ns | narrowed; still ~5% behind |
| `xof/256` | ~235.0 ns | 229.4 ns | 227.9 ns | tie |
| `xof/1024` | ~899.9 ns | 883.2 ns | 903.3 ns | tie/slightly ahead |
| `xof/4096` | ~1.07 us | 1.11 us | 1.09 us | noisy; forced AVX-512 was 1.07 us |
| `xof/65536` | ~10.7 us | 10.7 us | 10.5 us | tie |
| `xof/262144` | ~33.6 us | 33.7 us | 40.3 us | still ahead |
| `xof/1048576` | noisy | 107.1 us | 163.1 us | still ahead; do not overfit |

Conclusion: keep the XOF-many route. It removes the short-XOF gap on 0/1/32
byte outputs, leaves 64 bytes as the next exact-block target, and does not show
a credible large-output regression.

### Linux x86_64 Follow-up: 256B Retargeted

Current Sapphire Rapids diagnostic rows no longer justify spending the next
iteration on the 256B keyed/oneshot path for this host. Keep the old AMD Zen5
and Intel Ice Lake losses in the matrix, but do not use Sapphire Rapids as
evidence for that target.

Command:

```bash
cargo bench --features 'blake3,parallel,diag' --bench blake3 -- blake3/keyed --sample-size 10 --warm-up-time 1 --measurement-time 1 --noplot
```

Observed on `linux-dev`:

| Row | Public rscrypto | Forced AVX-512 | Forced AVX2 | Upstream | Result |
| --- | ---: | ---: | ---: | ---: | --- |
| `oneshot/256` | 216.2 ns | 215.4 ns | 268.2 ns | 250.4 ns | rscrypto ahead |
| `keyed/256` | 216.4 ns | 221.2 ns | 274.9 ns | 233.9 ns | rscrypto ahead |

Conclusion: exact 256B is still a candidate for AMD Zen5 and Intel Ice Lake,
but not the next Sapphire Rapids ownership move. The current public path is
already better than upstream on this machine.

### Linux x86_64 Follow-up: Dead XOF ASM Removed

Removed unused upstream-derived x86 symbols from the assembly files:

- `rscrypto_blake3_compress_xof_sse41`
- `_rscrypto_blake3_compress_xof_sse41`
- `rscrypto_blake3_compress_xof_avx512`
- `_rscrypto_blake3_compress_xof_avx512`

`rg compress_xof src/hashes/crypto/blake3` now finds no live assembly labels or
Rust declarations, only comments mentioning the generic BLAKE3 concept. The
active AVX-512 `rscrypto_blake3_xof_many_avx512` entrypoint remains, because it
is used by the retained short-XOF path. This does not improve speed, but it
shrinks the vendored ASM surface by about 600 lines and removes dead exported
symbols from the ownership ledger.

Current-tree follow-up: the remaining AVX-512 `compress_xof` local labels were
removed from the Linux, macOS, and Windows assembly variants. The live
`compress_in_place_avx512` and `xof_many_avx512` exports remain.

Verification:

```bash
rg -n "compress_xof_avx512|rscrypto_blake3_xof_many_avx512|rscrypto_blake3_compress_in_place_avx512" src/hashes/crypto/blake3/x86_64/asm.rs src/hashes/crypto/blake3/x86_64/asm/rscrypto_blake3_avx512_x86-64_*.s
cargo check --features 'blake3,diag,parallel' --bench blake3
scripts/check/asm-ledger.sh
git diff --check
mutagen sync flush rscrypto-linux
ssh ubuntu@linux-dev 'source ~/.cargo/env && cd /home/ubuntu/rscrypto && cargo check --features "blake3,diag,parallel" --bench blake3 && cargo test --features "blake3,diag,parallel" hashes::crypto::blake3::tests::diag_forced_kernels_match_normal_paths'
```

No `ct.toml` update is needed for this slice. It removes unreachable assembly
and does not change dispatch, keyed behavior, or any secret boundary.

### Ownership Slice: SSE4.1 Compress-In-Place Removed

Selected the SSE4.1 `compress_in_place` entrypoint as the first live ownership
move. It was the lowest-risk upstream ASM removal because rscrypto already had
an owned Rust intrinsic equivalent:

- owned replacement: `x86_64::compress_in_place_sse41_bytes`
- removed symbol: `rscrypto_blake3_compress_in_place_sse41`
- deleted files: `src/hashes/crypto/blake3/x86_64/asm/rscrypto_blake3_sse41_x86-64_*.s`

Selection criteria:

- no public API change;
- no dispatch-policy change;
- no new key-dependent branch or secret boundary;
- forced-kernel diagnostics can compare SSE4.1 against the normal path;
- the ASM ledger must pass after the files are deleted.

This shifts live SSE4.1 single-block/chunk compression to owned Rust intrinsics
instead of upstream-derived assembly. AVX2 and AVX-512 assembly remain in the
external-derived bucket.

Verification:

- macOS: `cargo check --features 'blake3,diag,parallel' --bench blake3`
- macOS: `cargo test --features 'blake3,diag,parallel' hashes::crypto::blake3::tests::diag_forced_kernels_match_normal_paths`
- Linux Sapphire Rapids: same check and forced-kernel test through `just ssh-linux`
- Ledger: `scripts/check/asm-ledger.sh` reports `68 assembly files, 13 owned, 55 external-derived, 0 candidates`

Small Linux forced-SSE4.1 spot checks after the replacement:

| Row | Median | Criterion change vs local baseline | Result |
| --- | ---: | ---: | --- |
| `blake3/rscrypto-x86-sse41/64` | 53.880 ns | -15.935% | improved |
| `blake3/keyed/rscrypto-x86-sse41/64` | 55.945 ns | -11.886% | improved |

Conclusion: this is not only a provenance cleanup. On the Sapphire Rapids host,
the owned Rust intrinsic SSE4.1 path beats the prior local Criterion baseline
for 64-byte forced-SSE4.1 digest/keyed rows. Keep the deletion.

No `ct.toml` update is needed for this slice. The replacement preserves the
same BLAKE3 compression semantics and does not add a key-dependent fast path or
new secret boundary.

### Ownership Slice: AVX2 Compress-In-Place Removed

Removed the live AVX2 `compress_in_place` assembly symbol and routed callers
through the owned Rust intrinsic compressor:

- owned replacement: `x86_64::compress_in_place_avx2_bytes`
- removed symbol: `rscrypto_blake3_compress_in_place_avx2`
- retained AVX2 assembly: `rscrypto_blake3_hash_many_avx2`

The AVX2 assembly files remain external-derived because `hash_many_avx2` is
still live, but the obsolete `compress_in_place_avx2` tail body is gone from the
Linux, macOS, and Windows files. This removed about 285 lines of upstream-derived
AVX2 assembly without changing dispatch policy.

Verification:

- macOS: `cargo check --features 'blake3,diag,parallel' --bench blake3`
- macOS: `cargo test --features 'blake3,diag,parallel' hashes::crypto::blake3::tests::diag_forced_kernels_match_normal_paths`
- Linux Sapphire Rapids: same check and forced-kernel test after `mutagen sync flush rscrypto-linux`

Small Linux forced-AVX2 spot checks after the replacement:

| Row | Median | Criterion change vs local baseline | Result |
| --- | ---: | ---: | --- |
| `blake3/rscrypto-x86-avx2/64` | 55.286 ns | -11.895% | improved |
| `blake3/keyed/rscrypto-x86-avx2/64` | 56.870 ns | -11.425% | improved |

Conclusion: keep the AVX2 compress-in-place deletion. The local intrinsic path
is both owned and faster on the tested Sapphire Rapids short-input rows.

No `ct.toml` update is needed for this slice. As with SSE4.1, this replaces an
implementation of the same compression primitive and does not add a new
key-dependent decision point.

### Rejected Slice: AVX-512 Compress-In-Place Intrinsics

Tried the same ownership move for AVX-512 `compress_in_place`: route callsites
through `x86_64::compress_in_place_avx512_bytes`, remove
`rscrypto_blake3_compress_in_place_avx512`, and leave `hash_many_avx512` plus
`xof_many_avx512` assembly intact.

Correctness passed:

- macOS: `cargo check --features 'blake3,diag,parallel' --bench blake3`
- macOS: `cargo test --features 'blake3,diag,parallel' hashes::crypto::blake3::tests::diag_forced_kernels_match_normal_paths`
- Linux Sapphire Rapids: same check and forced-kernel test after `mutagen sync flush rscrypto-linux`

But the affected forced-AVX-512 short-input rows regressed on Linux:

| Row | Median | Criterion change vs local baseline | Result |
| --- | ---: | ---: | --- |
| `blake3/rscrypto-x86-avx512/1` | 91.122 ns | +17.278% | rejected |
| `blake3/rscrypto-x86-avx512/32` | 68.775 ns | +30.009% | rejected |
| `blake3/keyed/rscrypto-x86-avx512/1` | 81.313 ns | +4.202% | rejected |
| `blake3/keyed/rscrypto-x86-avx512/32` | 68.136 ns | +31.498% | rejected |

Conclusion: keep the AVX-512 assembly `compress_in_place` symbol for now. The
current owned intrinsic fallback is correct but not fast enough for the
short-input rows that actually use it. Owning this path requires an rscrypto
AVX-512-specific compressor that keeps the fast rotate/codegen properties of
the assembly path, not a straight switch to the existing generic intrinsic
fallback.

No `ct.toml` update is needed. The rejected change was reverted and no new
secret boundary remains.

### Diagnostic Slice: AVX-512 Owned-Compress Selector

Added a diagnostic-only selector:

- label: `x86-avx512-owned-compress`
- kernel table override: owned AVX-512 CV compression and chunk compression
- retained paths: AVX-512 `hash_many` assembly and AVX-512 `xof_many` assembly
- streaming support: disabled, because streaming `ChunkState` still carries only
  `Blake3KernelId`, not a full diagnostic `Kernel` table

The selector intentionally disables the x86 exact-block one-chunk `hash_many`
shortcut. That makes one-shot digest/keyed rows exercise the owned AVX-512
compressor instead of silently routing through `hash_many_avx512`.

Correctness passed:

```bash
cargo check --features 'blake3,diag,parallel' --bench blake3
cargo test --features 'blake3,diag,parallel' hashes::crypto::blake3::tests::diag_forced_kernels_match_normal_paths
mutagen sync flush rscrypto-linux
ssh ubuntu@linux-dev 'source ~/.cargo/env && cd /home/ubuntu/rscrypto && cargo check --features "blake3,diag,parallel" --bench blake3 && cargo test --features "blake3,diag,parallel" hashes::crypto::blake3::tests::diag_forced_kernels_match_normal_paths'
```

Focused Sapphire Rapids benchmark:

```bash
ssh ubuntu@linux-dev 'source ~/.cargo/env && cd /home/ubuntu/rscrypto && cargo bench --features "blake3,diag,parallel" --bench blake3 -- "^(blake3|blake3/keyed|blake3/xof)/(rscrypto-x86-avx512|rscrypto-x86-avx512-owned-compress|blake3)/(0|1|32|64|256|1024)$" --warm-up-time 2 --measurement-time 4 --sample-size 30'
```

Digest/keyed rows:

| Row | Current AVX-512 | Owned-compress diag | Decision |
| --- | ---: | ---: | --- |
| `blake3/0` | 52.484 ns | 54.740 ns | reject |
| `blake3/1` | 80.671 ns | 84.649 ns | reject |
| `blake3/32` | 52.700 ns | 56.558 ns | reject |
| `blake3/64` | 52.542 ns | 54.994 ns | reject |
| `blake3/256` | 216.30 ns | 242.45 ns | reject |
| `blake3/1024` | 877.94 ns | 977.15 ns | reject |
| `keyed/0` | 51.903 ns | 54.440 ns | reject |
| `keyed/1` | 82.365 ns | 85.433 ns | reject |
| `keyed/32` | 53.028 ns | 57.258 ns | reject |
| `keyed/64` | 52.960 ns | 55.824 ns | reject |
| `keyed/256` | 218.01 ns | 241.25 ns | reject |
| `keyed/1024` | 867.64 ns | 956.24 ns | reject |

XOF rows:

| Row | Current AVX-512 | Owned-compress diag | Decision |
| --- | ---: | ---: | --- |
| `xof/0` | 51.018 ns | 51.989 ns | flat/slower |
| `xof/1` | 78.061 ns | 79.183 ns | flat/slower |
| `xof/32` | 51.681 ns | 50.874 ns | noise; not a compress signal |
| `xof/64` | 57.801 ns | 59.577 ns | reject |
| `xof/256` | 224.63 ns | 269.25 ns | reject |
| `xof/1024` | 877.08 ns | 1.0678 us | reject |

Conclusion: keep the diagnostic selector, but do not promote the current owned
AVX-512 intrinsic compressor. This confirms the earlier broad rejection with a
cleaner measurement boundary. Owning AVX-512 `compress_in_place` still requires
a new rscrypto-owned fast compressor, likely assembly or a materially different
schedule, not this fallback.

No `ct.toml` update is needed. The new selector is diagnostic-only, follows the
same public-length dispatch boundaries, and does not add a new key-dependent
decision point.

### Ownership Slice: AVX2 Full-Batch Hash-Many Promoted

Added `diag`-only owned hash-many candidates:

- `x86-avx2-owned-hash-many`
- `x86-avx512-owned-hash-many`

These candidates let the existing Rust intrinsic fallbacks run on
Linux/macOS/Windows without changing normal dispatch first. Streaming diagnostics
intentionally skip these variants for now, because the streaming API currently
forces only a kernel id through `update_with`; it does not carry a custom
`Kernel` function-pointer table end to end.

Linux Sapphire Rapids diagnostic result:

- AVX2 owned full 8-lane batches are materially faster than the AVX2 assembly
  `hash_many` path.
- The first broad AVX2 owned-tail check showed the exact 4-chunk row is slower
  than the AVX2 assembly tail. Later per-degree diagnostics split this by tail
  size and found a narrower 3/5/6/7-chunk win; see the selective tail slice
  below.
- AVX-512 owned hash-many is slower than AVX-512 assembly across tail and full
  16-lane sizes, so it stays diagnostic-only.

Promoted only the safe part: `hash_many_contiguous_avx2_wrapper` now calls the
owned `avx2::hash8_owned` helper inside the `num_chunks >= 8` loop, while
sub-degree tails and parent reductions keep using AVX2 assembly.

Post-promotion Linux spot checks:

| Row | Median | Criterion change vs prior local baseline | Result |
| --- | ---: | ---: | --- |
| `blake3/rscrypto-x86-avx2/16384` | 3.6666 us | -19.871% | improved |
| `blake3/keyed/rscrypto-x86-avx2/16384` | 3.7453 us | -17.796% | improved |
| `blake3/streaming/rscrypto-x86-avx2/16384B` | 244.63 us | -18.403% | improved |
| `blake3/xof/rscrypto-x86-avx2/16384` | 3.6895 us | -22.279% | improved |
| `blake3/rscrypto-x86-avx2/4096` | 1.5630 us | -4.7907% | tail intact |
| `blake3/keyed/rscrypto-x86-avx2/4096` | 1.5895 us | -1.1878% | flat |
| `blake3/streaming/rscrypto-x86-avx2/4096B` | 420.80 us | -2.7111% | tail intact |
| `blake3/xof/rscrypto-x86-avx2/4096` | 1.5925 us | +0.9336% | flat |

Rejected AVX-512 hash-many diagnostic spot checks:

| Row | ASM Median | Owned Median | Result |
| --- | ---: | ---: | --- |
| `blake3/rscrypto-x86-avx512/4096` | 1.0446 us | 2.9559 us | rejected |
| `blake3/rscrypto-x86-avx512/16384` | 2.7066 us | 2.9065 us | rejected |
| `blake3/rscrypto-x86-avx512/65536` | 10.434 us | 11.818 us | rejected |
| `blake3/rscrypto-x86-avx512/262144` | 34.531 us | 36.236 us | rejected |
| `blake3/rscrypto-x86-avx512/1048576` | 158.42 us | 176.25 us | rejected |

Verification:

```bash
cargo fmt --all
cargo check --features 'blake3,diag,parallel' --bench blake3
cargo test --features 'blake3,diag,parallel' hashes::crypto::blake3::tests::diag_forced_kernels_match_normal_paths
mutagen sync flush rscrypto-linux
ssh ubuntu@linux-dev-sync 'bash -lc "cd /home/ubuntu/rscrypto && cargo check --features '\''blake3,diag,parallel'\'' --bench blake3"'
ssh ubuntu@linux-dev-sync 'bash -lc "cd /home/ubuntu/rscrypto && cargo test --features '\''blake3,diag,parallel'\'' hashes::crypto::blake3::tests::diag_forced_kernels_match_normal_paths"'
```

No `ct.toml` update is needed for this slice. This changes the implementation
of the same public keyed/unkeyed BLAKE3 paths and does not add a new
key-dependent branch or secret boundary. The existing keyed BLAKE3 CT coverage
still exercises the public API surface.

### Ownership Slice: AVX2 Full Parent Batches Promoted

Moved the AVX2 parent-CV reduction path one step further toward ownership:

- full 8-parent batches now call owned `avx2::hash8_owned`;
- at this point, partial parent tails still used `hash_many_avx2` assembly;
- the now-dead supported-OS `avx2::hash8` wrapper that only forwarded to
  assembly was removed.

This mirrors the previous contiguous-chunk decision: owned AVX2 is strong when
all 8 lanes are real work, while duplicate-lane tails are not good enough yet.

Linux Sapphire Rapids post-change spot checks:

| Row | Median | Criterion change vs prior local baseline | Result |
| --- | ---: | ---: | --- |
| `blake3/rscrypto-x86-avx2/65536` | 14.573 us | -18.731% | improved |
| `blake3/keyed/rscrypto-x86-avx2/65536` | 14.600 us | -18.491% | improved |
| `blake3/streaming/rscrypto-x86-avx2/65536B` | 233.32 us | -18.875% | improved |
| `blake3/xof/rscrypto-x86-avx2/65536` | 14.657 us | -17.854% | improved |
| `blake3/rscrypto-x86-avx2/262144` | 39.134 us | -17.684% | improved |
| `blake3/keyed/rscrypto-x86-avx2/262144` | 39.240 us | -18.109% | improved |
| `blake3/xof/rscrypto-x86-avx2/262144` | 39.225 us | -17.751% | improved |
| `blake3/rscrypto-x86-avx2/1048576` | 220.10 us | -17.270% | improved |
| `blake3/keyed/rscrypto-x86-avx2/1048576` | 215.63 us | -22.670% | improved |
| `blake3/xof/rscrypto-x86-avx2/1048576` | 221.82 us | -20.311% | improved |

Verification:

```bash
cargo fmt --all
cargo check --features 'blake3,diag,parallel' --bench blake3
cargo test --features 'blake3,diag,parallel' hashes::crypto::blake3::tests::diag_forced_kernels_match_normal_paths
mutagen sync flush rscrypto-linux
ssh ubuntu@linux-dev-sync 'bash -lc "cd /home/ubuntu/rscrypto && cargo check --features '\''blake3,diag,parallel'\'' --bench blake3"'
ssh ubuntu@linux-dev-sync 'bash -lc "cd /home/ubuntu/rscrypto && cargo test --features '\''blake3,diag,parallel'\'' hashes::crypto::blake3::tests::diag_forced_kernels_match_normal_paths"'
```

No `ct.toml` update is needed. This is still the same public BLAKE3 reduction
tree with the same key schedule and no new key-dependent branch.

### Ownership Slice: AVX2 Exact-Block One-Chunk Path Promoted

Removed the direct AVX2 `hash_many` assembly calls from the exact-block
one-chunk fast paths in `mod.rs`.

The promoted owned path is a dependency-chained AVX2 CV compressor loop:

- digest/keyed exact-block one-chunk inputs now use owned AVX2 compression;
- XOF exact-block prefix compression now uses the same owned AVX2 chain;
- the AVX-512 256B heuristic still chooses AVX2, but now chooses the owned AVX2
  chain instead of AVX2 assembly;
- the temporary `x86-avx2-owned-exact-blocks` diagnostic selector was removed
  after promotion to avoid permanent benchmark-only dispatch state.

This removes all direct `hash_many_avx2` callsites from `mod.rs`. Remaining
AVX2 assembly use is now limited to `kernels.rs` partial tails and the FFI
wrapper.

Linux Sapphire Rapids candidate measurement before promotion:

| Row | Current AVX2 ASM median | Owned candidate median | Result |
| --- | ---: | ---: | --- |
| `blake3/rscrypto-x86-avx2/64` | 55.621 ns | 54.968 ns | slight win |
| `blake3/rscrypto-x86-avx2/256` | 255.00 ns | 251.54 ns | slight win |
| `blake3/rscrypto-x86-avx2/1024` | 1.0504 us | 1.0514 us | flat |
| `blake3/keyed/rscrypto-x86-avx2/256` | 252.19 ns | 250.97 ns | slight win |
| `blake3/xof/rscrypto-x86-avx2/256` | 278.89 ns | 279.57 ns | flat |
| `blake3/xof/rscrypto-x86-avx2/1024` | 1.0761 us | 1.0755 us | flat |

Linux Sapphire Rapids post-promotion isolated AVX2 rerun:

| Row | Median | Criterion change | Result |
| --- | ---: | ---: | --- |
| `blake3/rscrypto-x86-avx2/64` | 54.675 ns | -0.272% | flat |
| `blake3/rscrypto-x86-avx2/256` | 249.41 ns | -1.990% | improved |
| `blake3/rscrypto-x86-avx2/1024` | 1.0385 us | -1.334% | improved |
| `blake3/keyed/rscrypto-x86-avx2/64` | 55.346 ns | -2.511% | improved |
| `blake3/keyed/rscrypto-x86-avx2/256` | 249.93 ns | -2.857% | improved |
| `blake3/keyed/rscrypto-x86-avx2/1024` | 1.0377 us | -2.977% | improved |
| `blake3/xof/rscrypto-x86-avx2/64` | 69.988 ns | -2.586% | improved |
| `blake3/xof/rscrypto-x86-avx2/256` | 277.59 ns | -2.772% | improved |
| `blake3/xof/rscrypto-x86-avx2/1024` | 1.0761 us | -2.572% | improved |

The mixed AVX2/AVX-512 production pass was noisy enough to show unrelated
regressions on rows that did not change. The isolated AVX2 rerun above is the
decision data for keeping the promotion.

Verification:

```bash
cargo fmt --all
cargo check --features 'blake3,diag,parallel' --bench blake3
cargo test --features 'blake3,diag,parallel' hashes::crypto::blake3::tests::diag_forced_kernels_match_normal_paths
mutagen sync flush rscrypto-linux
ssh ubuntu@linux-dev-sync 'bash -lc "cd /home/ubuntu/rscrypto && cargo check --features '\''blake3,diag,parallel'\'' --bench blake3"'
ssh ubuntu@linux-dev-sync 'bash -lc "cd /home/ubuntu/rscrypto && cargo test --features '\''blake3,diag,parallel'\'' hashes::crypto::blake3::tests::diag_forced_kernels_match_normal_paths"'
ssh ubuntu@linux-dev-sync 'bash -lc "cd /home/ubuntu/rscrypto && cargo bench --features '\''blake3,diag,parallel'\'' --bench blake3 -- '\''(blake3|blake3/keyed|blake3/xof)/rscrypto-x86-avx2/(64|256|1024)'\'' --warm-up-time 2 --measurement-time 4 --sample-size 30"'
```

No `ct.toml` update is needed. The branch is selected by public input length
and CPU capability, uses the same keyed/unkeyed compression semantics, and
does not add a key-dependent path.

### Ownership Slice: AVX2 Serial Tail Candidate Rejected

Candidate: replace the remaining AVX2 contiguous sub-degree tail ASM with a
diagnostic-only owned serial tail. The candidate kept the owned 8-lane full
batches, then hashed each 1 KiB tail chunk with the owned AVX2 single-block
compressor loop.

Correctness passed locally and on Linux:

```bash
cargo check --features 'blake3,diag,parallel' --bench blake3
cargo test --features 'blake3,diag,parallel' hashes::crypto::blake3::tests::diag_forced_kernels_match_normal_paths
ssh ubuntu@linux-dev 'bash -lc "cd /home/ubuntu/rscrypto && cargo check --features \"blake3,diag,parallel\" --bench blake3 && cargo test --features \"blake3,diag,parallel\" hashes::crypto::blake3::tests::diag_forced_kernels_match_normal_paths"'
```

Linux target: Intel Xeon Platinum 8488C, Sapphire Rapids, 8 vCPU KVM guest.
Focused Criterion command:

```bash
cargo bench --features 'blake3,diag,parallel' --bench blake3 -- '^(blake3|blake3/keyed|blake3/xof)/rscrypto-x86-avx2(-owned-hash-many|-owned-tail-serial)?/4096$' --warm-up-time 2 --measurement-time 4 --sample-size 30
```

| Row | Current AVX2 ASM tail | Owned duplicate-lane tail | Owned serial tail | Decision |
| --- | ---: | ---: | ---: | --- |
| `blake3/4096` | 1.5730 us | 1.8106 us | 4.0918 us | reject serial |
| `blake3/keyed/4096` | 1.5818 us | 1.8084 us | 4.0869 us | reject serial |
| `blake3/xof/4096` | 1.5791 us | 1.8148 us | 4.0987 us | reject serial |

The serial tail is correct but more than 2.5x slower than the current ASM tail
on the pure four-chunk tail row. The duplicate-lane owned diagnostic path is
also slower than ASM here. No serial-tail diagnostic code was retained; the
notebook entry is the artifact.

No `ct.toml` update is needed. The experiment changed only public-length tail
dispatch and introduced no key-dependent branch or new secret boundary.

### Ownership Slice: AVX2 Serial Parent Tail Rejected

Candidate: replace AVX2 partial parent-tail ASM (`rem < 8` parent pairs) with
serial owned AVX2 `parent_cv_avx2` calls. Full 8-parent batches stayed on the
already-promoted owned `hash8_owned` path.

Correctness passed locally and on Linux:

```bash
cargo check --features 'blake3,diag,parallel' --bench blake3
cargo test --features 'blake3,diag,parallel' hashes::crypto::blake3::tests::diag_forced_kernels_match_normal_paths
mutagen sync flush rscrypto-linux
ssh ubuntu@linux-dev 'bash -lc "cd /home/ubuntu/rscrypto && cargo check --features \"blake3,diag,parallel\" --bench blake3 && cargo test --features \"blake3,diag,parallel\" hashes::crypto::blake3::tests::diag_forced_kernels_match_normal_paths"'
```

Focused Linux Sapphire Rapids benchmark:

```bash
cargo bench --features 'blake3,diag,parallel' --bench blake3 -- '^(blake3|blake3/keyed|blake3/xof)/rscrypto-x86-avx2/4096$' --warm-up-time 2 --measurement-time 4 --sample-size 30
```

| Row | Serial parent-tail median | Criterion change vs ASM parent tail | Decision |
| --- | ---: | ---: | --- |
| `blake3/4096` | 1.6081 us | +2.2846% | reject |
| `blake3/keyed/4096` | 1.6084 us | +2.0285% | reject |
| `blake3/xof/4096` | 1.6131 us | +2.1224% | reject |

The generic AVX2 `hash_many` ASM remains faster for partial parent batches on
Sapphire Rapids. The experiment was reverted; only full 8-parent batches remain
owned in production.

No `ct.toml` update is needed. The experiment changed only public tree-shape
dispatch and introduced no key-dependent branch or new secret boundary.

### Rejected Slice: AVX2 Lower-Width Tail Fallbacks

Candidate: use the already-owned SSE4.1 `hash4` kernel as a lower-width fallback
for AVX2 sub-degree work, instead of keeping the current AVX2 `hash_many` ASM
tail. This was checked in two steps:

1. First, measure forced SSE4.1 at the pure 4-chunk row (`4096`) against forced
   AVX2 and AVX-512 to see whether a chunk-tail handoff was even credible.
2. Then, temporarily route only AVX2 parent tails with `rem <= 4` through
   owned SSE4.1 `hash4`, leaving AVX2 chunk tails unchanged.

Sapphire Rapids baseline check:

```bash
mutagen sync flush rscrypto-linux
ssh ubuntu@linux-dev 'source ~/.cargo/env && cd /home/ubuntu/rscrypto && cargo bench --features "blake3,diag,parallel" --bench blake3 -- "^(blake3|blake3/keyed|blake3/xof)/(rscrypto-x86-sse41|rscrypto-x86-avx2|rscrypto-x86-avx512|blake3)/4096$" --warm-up-time 2 --measurement-time 4 --sample-size 30'
```

| Row | Forced SSE4.1 | Forced AVX2 | Forced AVX-512 | Upstream | Decision |
| --- | ---: | ---: | ---: | ---: | --- |
| `blake3/4096` | 1.7377 us | 1.6121 us | 1.0444 us | 1.0643 us | reject SSE4 chunk tail |
| `blake3/keyed/4096` | 1.7623 us | 1.5859 us | 1.0692 us | 1.0755 us | reject SSE4 chunk tail |
| `blake3/xof/4096` | 1.7560 us | 1.6113 us | 1.0686 us | 1.0791 us | reject SSE4 chunk tail |

The parent-tail-only prototype was byte-correct but slower:

```bash
mutagen sync flush rscrypto-linux
ssh ubuntu@linux-dev 'source ~/.cargo/env && cd /home/ubuntu/rscrypto && cargo check --features "blake3,diag,parallel" --bench blake3 && cargo test --features "blake3,diag,parallel" hashes::crypto::blake3::tests::diag_forced_kernels_match_normal_paths && cargo bench --features "blake3,diag,parallel" --bench blake3 -- "^(blake3|blake3/keyed|blake3/xof)/rscrypto-x86-avx2/4096$" --warm-up-time 2 --measurement-time 4 --sample-size 30'
```

| Row | Parent-tail SSE4.1 median | Criterion change vs AVX2 ASM parent tail | Decision |
| --- | ---: | ---: | --- |
| `blake3/rscrypto-x86-avx2/4096` | 1.6766 us | +4.6656% | reject |
| `blake3/keyed/rscrypto-x86-avx2/4096` | 1.6721 us | +3.5054% | reject |
| `blake3/xof/rscrypto-x86-avx2/4096` | 1.6932 us | +5.5717% | reject |

Conclusion: do not replace AVX2 tails with owned SSE4.1 fallback logic. The
remaining AVX2 tail ownership problem needs a real AVX2 tail-specific kernel or
ASM rewrite, not a cascade to the existing lower-width kernel.

No `ct.toml` update is needed. The rejected prototype changed only public
tree-shape dispatch and was reverted.

### Rejected Slice: AVX2 VEX-Coded 4-Lane Tail

Candidate: add an owned 4-lane `hash_many` tail in `x86_64::avx2` by copying
the owned SSE4.1 4-way structure into an AVX2 target-feature function. The
hypothesis was that VEX-encoded 128-bit operations might avoid the AVX/SSE
handoff cost while avoiding the wasted lanes from the duplicate-lane 8-way tail.

The prototype routed only exact 4-lane AVX2 tails through the new owned helper:

- contiguous chunk tails with `num_chunks == 4`;
- partial parent tails with `rem == 4`;
- all 1/2/3/5/6/7 AVX2 tails stayed on the existing assembly path.

Correctness passed locally and on Sapphire Rapids:

```bash
cargo check --features 'blake3,diag,parallel' --bench blake3
mutagen sync flush rscrypto-linux
ssh ubuntu@linux-dev 'source ~/.cargo/env && cd /home/ubuntu/rscrypto && cargo check --features "blake3,diag,parallel" --bench blake3 && cargo test --features "blake3,diag,parallel" hashes::crypto::blake3::tests::diag_forced_kernels_match_normal_paths'
```

Focused Sapphire Rapids benchmark:

```bash
ssh ubuntu@linux-dev 'source ~/.cargo/env && cd /home/ubuntu/rscrypto && cargo bench --features "blake3,diag,parallel" --bench blake3 -- "^(blake3|blake3/keyed|blake3/xof)/(rscrypto-x86-avx2|blake3)/(4096|8192)$|^blake3/streaming/(rscrypto-x86-avx2|blake3)/4096B$" --warm-up-time 2 --measurement-time 4 --sample-size 30'
```

| Row | AVX2 4-lane owned-tail median | Criterion change vs AVX2 ASM tail | Decision |
| --- | ---: | ---: | --- |
| `blake3/rscrypto-x86-avx2/4096` | 1.6903 us | +1.5046% | reject |
| `blake3/keyed/rscrypto-x86-avx2/4096` | 1.6979 us | +3.4684% | reject |
| `blake3/streaming/rscrypto-x86-avx2/4096B` | 452.67 us | +7.7321% | reject |
| `blake3/xof/rscrypto-x86-avx2/4096` | 1.7013 us | +0.7552% | flat/no win |

The prototype was reverted. VEX-encoding the current 4-way Rust shape is still
not enough; the upstream-derived AVX2 tail wins because it has a genuinely
specialized 4/2/1-lane schedule. The next owned candidate needs a new
tail-specific AVX2 schedule or rscrypto-owned ASM, not another wrapper around
the existing 4-way structure.

No `ct.toml` update is needed. This touched only public-length tail dispatch and
was reverted.

### Ownership Slice: AVX2 Selective Chunk Tails Promoted

Added raw diagnostic helpers and benchmarks so contiguous chunk-CV tails and
parent-CV tails can be measured directly instead of inferred from public digest
rows:

- `diag_blake3_chunk_cvs_with_kernel`
- `diag_blake3_parent_cvs_with_kernel`
- `blake3/chunk-tail-digest`
- `blake3/chunk-tail-cvs`
- `blake3/parent-tail-cvs`

The raw chunk-CV sweep showed the broad "duplicate lanes are bad" conclusion
was too coarse. Duplicate-lane AVX2 is bad for 1/2/4 chunk tails, but clearly
better for 3/5/6/7 on Sapphire Rapids. This slice routed only those winning
contiguous chunk-tail counts through the owned `avx2::hash8_owned` duplicate
lane helper. The follow-up one-chunk slice below promotes the `1` tail through a
different owned serial AVX2 route; `2/4` remain on `hash_many_avx2` assembly.

Pre-promotion raw chunk-CV medians on `linux-dev`:

| Chunks | AVX2 ASM tail | Owned duplicate tail | Decision |
| ---: | ---: | ---: | --- |
| 1 | 1.0723 us | 1.6899 us | keep ASM |
| 2 | 1.0840 us | 1.6797 us | keep ASM |
| 3 | 2.1766 us | 1.6813 us | promote owned |
| 4 | 1.4330 us | 1.6648 us | keep ASM |
| 5 | 2.4948 us | 1.6603 us | promote owned |
| 6 | 2.5389 us | 1.6783 us | promote owned |
| 7 | 3.6517 us | 1.7060 us | promote owned |
| 8 | 1.6571 us | 1.6560 us | already full owned |
| 15 | 5.2200 us | 3.3323 us | improves via 8+7 |

Post-promotion raw chunk-CV medians:

| Chunks | Production AVX2 | Owned diagnostic | Criterion change vs old AVX2 | Result |
| ---: | ---: | ---: | ---: | --- |
| 1 | 1.0719 us | 1.6636 us | no change | ASM preserved |
| 2 | 1.0630 us | 1.6909 us | -2.2222% | ASM preserved |
| 3 | 1.6824 us | 1.6777 us | -22.379% | improved |
| 4 | 1.4215 us | 1.6724 us | noise | ASM preserved |
| 5 | 1.6619 us | 1.6743 us | -33.137% | improved |
| 6 | 1.6625 us | 1.6845 us | -34.302% | improved |
| 7 | 1.6706 us | 1.6809 us | -53.622% | improved |
| 8 | 1.6480 us | 1.6496 us | noise | full owned |
| 15 | 3.3030 us | 3.3499 us | -36.856% | improved |

The public digest tail benchmark confirms the routing survives the higher-level
tree path:

| Chunks | Production AVX2 | Owned diagnostic | Upstream `blake3` | Result |
| ---: | ---: | ---: | ---: | --- |
| 1 | 1.0740 us | 1.0759 us | 943.05 ns | flat |
| 2 | 1.1693 us | 1.7420 us | 956.42 ns | ASM preserved |
| 3 | 1.8508 us | 1.8449 us | 1.8761 us | matches owned |
| 4 | 1.6287 us | 1.8135 us | 1.0627 us | ASM preserved |
| 5 | 1.9209 us | 1.9172 us | 1.9972 us | matches owned |
| 6 | 1.9911 us | 1.9896 us | 2.0406 us | matches owned |
| 7 | 2.0971 us | 2.0630 us | 3.0023 us | matches owned/noisy |
| 8 | 1.9358 us | 1.9534 us | 1.8530 us | full owned |
| 15 | 4.1886 us | 4.1527 us | 4.6490 us | 8+7 improves |

Aggregate 4096-byte AVX2 sanity rows also improved:

| Row | Median | Criterion change | Result |
| --- | ---: | ---: | --- |
| `blake3/rscrypto-x86-avx2/4096` | 1.6008 us | -5.2792% | improved |
| `blake3/keyed/rscrypto-x86-avx2/4096` | 1.5789 us | -7.1048% | improved |
| `blake3/streaming/rscrypto-x86-avx2/4096B` | 421.20 us | -5.9910% | improved |
| `blake3/xof/rscrypto-x86-avx2/4096` | 1.5995 us | -6.7800% | improved |

Parent-tail diagnostics were added and baseline-measured in this slice. The
follow-up selective parent-tail promotion is recorded below.

Verification:

```bash
cargo fmt --all
cargo check --features 'blake3,diag,parallel' --bench blake3
cargo test --features 'blake3,diag,parallel' hashes::crypto::blake3::tests::diag_raw_cv_helpers_match_portable
cargo test --features 'blake3,diag,parallel' hashes::crypto::blake3::tests::diag_forced_kernels_match_normal_paths
mutagen sync flush rscrypto-linux
ssh ubuntu@linux-dev 'source ~/.cargo/env && cd /home/ubuntu/rscrypto && cargo check --features "blake3,diag,parallel" --bench blake3 && cargo test --features "blake3,diag,parallel" hashes::crypto::blake3::tests::diag_raw_cv_helpers_match_portable && cargo test --features "blake3,diag,parallel" hashes::crypto::blake3::tests::diag_forced_kernels_match_normal_paths'
ssh ubuntu@linux-dev 'source ~/.cargo/env && cd /home/ubuntu/rscrypto && cargo bench --features "blake3,diag,parallel" --bench blake3 -- "^blake3/(chunk-tail-cvs|chunk-tail-digest)/(rscrypto-x86-(avx2|avx2-owned-hash-many)|blake3)/(1|2|3|4|5|6|7|8|15)$" --warm-up-time 1 --measurement-time 2 --sample-size 20'
ssh ubuntu@linux-dev 'source ~/.cargo/env && cd /home/ubuntu/rscrypto && cargo bench --features "blake3,diag,parallel" --bench blake3 -- "^(blake3|blake3/keyed|blake3/xof)/(rscrypto-x86-avx2|rscrypto-x86-avx2-owned-hash-many|blake3)/(4096|8192)$|^blake3/streaming/(rscrypto-x86-avx2|blake3)/4096B$" --warm-up-time 1 --measurement-time 2 --sample-size 20'
```

Note: the common benchmark matrix currently has 4096 and 16384 but not 8192, so
the aggregate command above only emitted 4096 rows. The new
`chunk-tail-digest` group covers the exact 8192-byte 8-chunk digest case.

No `ct.toml` update is needed. The new production branch is selected only by
public chunk count and CPU capability. It uses the same keyed/unkeyed BLAKE3
compression semantics and does not introduce a key-dependent branch or new
secret boundary.

### Ownership Slice: AVX2 One-Chunk Tail Promoted

Candidate: replace only the remaining contiguous one-chunk AVX2 ASM tail with
an owned serial AVX2 CV loop using `compress_cv_avx2_bytes`. This is not the
rejected broad serial-tail route from earlier. The rejected route tried to cover
larger sub-degree tails and lost badly; this slice isolates the exact `rem == 1`
case after full 8-lane batches have already been consumed.

Focused raw measurements on `linux-dev`:

| Row | AVX2 ASM tail | Owned serial one-chunk tail | Owned duplicate-lane tail | Decision |
| --- | ---: | ---: | ---: | --- |
| `chunk-tail-cvs/1` | 1.0750 us | 1.0505 us | 1.6496 us | promote serial |
| `chunk-tail-cvs/9` | 2.7841 us | 2.7440 us | n/a | promote serial |
| `chunk-tail-digest/9` | 3.2285 us | 3.1790 us | n/a | promote serial |

Post-promotion focused verification on `linux-dev`:

| Row | Production AVX2 | Owned duplicate-lane diagnostic | Upstream `blake3` | Result |
| --- | ---: | ---: | ---: | --- |
| `chunk-tail-cvs/1` | 1.0439 us | 1.6565 us | n/a | serial tail active/improved |
| `chunk-tail-cvs/2` | 1.0750 us | 1.6916 us | n/a | ASM preserved |
| `chunk-tail-cvs/4` | 1.4219 us | 1.6529 us | n/a | ASM preserved |
| `chunk-tail-cvs/9` | 2.6961 us | 3.3130 us | n/a | full batch + serial tail improved |
| `chunk-tail-digest/1` | 1.0494 us | 1.0626 us | 940.28 ns | improved/no public gap closure |
| `chunk-tail-digest/9` | 3.1641 us | 3.7070 us | 2.7638 us | improved, still behind upstream |

The public one-chunk digest row stayed flat because the exact one-chunk digest
fast path had already been owned separately; this change matters for
`hash_many` rows that have a full AVX2 batch plus a one-chunk tail. Production
AVX2 contiguous chunk-tail ownership after this slice was:

| Tail count | Route |
| ---: | --- |
| 1 | owned serial AVX2 CV loop |
| 2 | `hash_many_avx2` assembly |
| 3 | owned duplicate-lane AVX2 |
| 4 | `hash_many_avx2` assembly |
| 5/6/7 | owned duplicate-lane AVX2 |

The temporary `x86-avx2-serial-one-chunk-tail` diagnostic label was removed
after promotion. The standard `x86-avx2` diagnostic now measures the production
route. No `ct.toml` update is needed: the branch is selected by public chunk
count and CPU capability, with no key-dependent control flow.

### Ownership Slice: AVX2 Two-Chunk Tail Promoted

Candidate: replace the remaining exact two-chunk AVX2 ASM tail with an owned
two-lane AVX2 chunk reducer. It reuses the two-parent YMM-half schedule but
runs the full 16-block chunk compression loop for two contiguous chunks. This is
not a general sub-degree replacement: pair-plus-serial and repeated pair calls
lost for 3/4/5+ chunks.

Focused raw chunk-CV measurements on `linux-dev` before promotion:

| Chunks | Production AVX2 | Owned pair-chunk diagnostic | Decision |
| ---: | ---: | ---: | --- |
| 1 | 1.0526 us | 1.0588 us | keep serial one-chunk route |
| 2 | 1.0890 us | 960.85 ns | promote pair route |
| 3 | 1.6442 us | 2.0362 us | keep duplicate-lane route |
| 4 | 1.4523 us | 1.8996 us | keep ASM |
| 5 | 1.6633 us | 2.9195 us | keep duplicate-lane route |
| 8 | 1.6726 us | 3.8218 us | keep full owned 8-lane route |
| 15 | 3.3486 us | 7.7173 us | keep 8+7 owned route |

Post-promotion raw chunk-CV measurements on `linux-dev`:

| Chunks | Production AVX2 | Owned pair-chunk diagnostic | Result |
| ---: | ---: | ---: | --- |
| 1 | 1.0446 us | 1.0730 us | unchanged serial route |
| 2 | 976.10 ns | 955.06 ns | production improved ~10.4% |
| 3 | 1.6461 us | 2.0008 us | duplicate-lane route preserved |
| 4 | 1.4771 us | 1.8884 us | ASM preserved |
| 5 | 1.6619 us | 2.9623 us | duplicate-lane route preserved |

Production AVX2 contiguous chunk-tail ownership is now:

| Tail count | Route |
| ---: | --- |
| 1 | owned serial AVX2 CV loop |
| 2 | owned two-chunk AVX2 reducer |
| 3 | owned duplicate-lane AVX2 |
| 4 | `hash_many_avx2` assembly |
| 5/6/7 | owned duplicate-lane AVX2 |
| 8 | owned full AVX2 batch |

No `ct.toml` update is needed. The new branch is selected by public chunk
count and CPU capability, and the reducer uses the same keyed/unkeyed
compression inputs as the existing chunk-CV path.

### Ownership Slice: AVX2 Selective Parent Tails Promoted

Extended the raw parent-CV diagnostics so `X86Avx2OwnedHashMany` measures the
owned duplicate-lane AVX2 parent reducer directly. This isolates parent-node
reduction from public digest overhead and lets us decide by parent count.

The Sapphire Rapids sweep showed the same lesson as chunk tails, but with a
different threshold: duplicate-lane AVX2 is bad for 1/2/3/4 parent tails and
good for 5/6/7. This slice routed only 5/6/7 parent tails through owned
`avx2::hash8_owned`. The follow-up one-parent slice below promotes the `1`
tail through a different owned serial AVX2 route; parent tails 2/3/4 stay on
`hash_many_avx2` assembly. Full 8-parent batches were already owned.

Pre-promotion raw parent-CV medians on `linux-dev`:

| Parents | AVX2 ASM tail | Owned duplicate tail | Decision |
| ---: | ---: | ---: | --- |
| 1 | 69.871 ns | 149.10 ns | keep ASM |
| 2 | 72.435 ns | 147.23 ns | keep ASM |
| 3 | 132.77 ns | 145.57 ns | keep ASM |
| 4 | 107.46 ns | 144.80 ns | keep ASM |
| 5 | 163.90 ns | 144.55 ns | promote owned |
| 6 | 168.37 ns | 144.34 ns | promote owned |
| 7 | 229.35 ns | 146.37 ns | promote owned |
| 8 | 149.51 ns | 146.93 ns | already full owned |
| 15 | 377.20 ns | 289.00 ns | improves via 8+7 |

Post-promotion raw parent-CV medians:

| Parents | Production AVX2 | Owned diagnostic | Criterion change vs old AVX2 | Result |
| ---: | ---: | ---: | ---: | --- |
| 1 | 68.644 ns | 147.04 ns | -1.4439% | ASM preserved |
| 2 | 71.283 ns | 146.00 ns | -1.4581% | ASM preserved |
| 3 | 131.10 ns | 145.02 ns | -1.7006% | ASM preserved |
| 4 | 103.75 ns | 143.69 ns | -2.3037% | ASM preserved |
| 5 | 146.24 ns | 143.16 ns | -10.925% | improved |
| 6 | 147.13 ns | 144.04 ns | -12.870% | improved |
| 7 | 147.86 ns | 144.56 ns | -35.543% | improved |
| 8 | 149.42 ns | 145.21 ns | -0.3388% | full owned/noise |
| 15 | 292.01 ns | 288.25 ns | -22.547% | improved |

Verification:

```bash
cargo fmt --all
cargo check --features 'blake3,diag,parallel' --bench blake3
cargo test --features 'blake3,diag,parallel' hashes::crypto::blake3::tests::diag_raw_cv_helpers_match_portable
cargo test --features 'blake3,diag,parallel' hashes::crypto::blake3::tests::diag_forced_kernels_match_normal_paths
mutagen sync flush rscrypto-linux
ssh ubuntu@linux-dev 'source ~/.cargo/env && cd /home/ubuntu/rscrypto && cargo check --features "blake3,diag,parallel" --bench blake3 && cargo test --features "blake3,diag,parallel" hashes::crypto::blake3::tests::diag_raw_cv_helpers_match_portable && cargo test --features "blake3,diag,parallel" hashes::crypto::blake3::tests::diag_forced_kernels_match_normal_paths'
ssh ubuntu@linux-dev 'source ~/.cargo/env && cd /home/ubuntu/rscrypto && cargo bench --features "blake3,diag,parallel" --bench blake3 -- "^blake3/parent-tail-cvs/rscrypto-x86-(avx2|avx2-owned-hash-many)/(1|2|3|4|5|6|7|8|15)$" --warm-up-time 1 --measurement-time 2 --sample-size 20'
```

No `ct.toml` update is needed. The branch is selected by public parent count
and CPU capability. It uses the same keyed/unkeyed compression inputs as the
existing parent reducer and does not add a key-dependent branch or new secret
boundary.

### Ownership Slice: AVX2 One-Parent Tail Promoted

Candidate: replace only the remaining one-parent AVX2 ASM tail with an owned
serial parent reducer using `parent_cv_avx2`. This is not the rejected broad
serial parent-tail route. Counts 2/3/4 still need a better tail-specific
schedule or rscrypto-owned ASM.

Focused parent-CV measurements on `linux-dev`:

| Parents | AVX2 ASM tail | Owned serial parent tail | Owned duplicate tail | Decision |
| ---: | ---: | ---: | ---: | --- |
| 1 | 69.456 ns | 52.870 ns | 144.84 ns | promote serial |
| 2 | 71.991 ns | 102.21 ns | 150.96 ns | keep ASM |
| 3 | 132.73 ns | 154.50 ns | 147.70 ns | keep ASM |
| 4 | 104.09 ns | 205.30 ns | 145.39 ns | keep ASM |
| 5 | 146.24 ns | 253.31 ns | 143.16 ns | keep duplicate-lane |

Post-promotion production confirmation on `linux-dev`:

| Parents | Production AVX2 | Owned diagnostic | Criterion result |
| ---: | ---: | ---: | --- |
| 1 | 57.213 ns | 145.22 ns | production improved 17.652% vs old AVX2 |
| 2 | 70.789 ns | 145.50 ns | production ASM preserved |
| 3 | 135.29 ns | 146.77 ns | production ASM preserved |
| 4 | 103.57 ns | 146.76 ns | production ASM preserved |
| 5 | 145.89 ns | 146.20 ns | production duplicate-lane preserved |

After this slice, production AVX2 parent-tail ownership was:

| Parent count | Route |
| ---: | --- |
| 1 | owned serial AVX2 parent reducer |
| 2/3/4 | `hash_many_avx2` assembly |
| 5/6/7 | owned duplicate-lane AVX2 |
| 8 | owned full AVX2 batch |

The temporary `x86-avx2-serial-parent-tail` diagnostic label was removed after
promotion. The standard `x86-avx2` diagnostic now measures the production route.
No `ct.toml` update is needed: the branch is selected by public parent count and
CPU capability, with no key-dependent control flow.

### Ownership Slice: AVX2 Pair-Parent Tails Promoted

Candidate: replace the remaining two- and three-parent AVX2 ASM tails with an
owned two-parent reducer that runs two independent parent compressions in the
two 128-bit halves of one YMM register. A three-parent tail runs that pair
reducer for the first two parents and the already-promoted serial parent reducer
for the final parent. This is intentionally not used for four parents: the
existing AVX2 assembly route remains slightly faster there.

Focused parent-CV measurements on `linux-dev` before promotion:

| Parents | Production AVX2 | Owned pair-parent diagnostic | Decision |
| ---: | ---: | ---: | --- |
| 1 | 57.140 ns | 54.768 ns | keep existing serial production route |
| 2 | 70.343 ns | 52.882 ns | promote pair route |
| 3 | 132.79 ns | 103.98 ns | promote pair + serial route |
| 4 | 102.79 ns | 103.92 ns | keep ASM |
| 5 | 145.93 ns | 154.95 ns | keep duplicate-lane route |

Important false start: routing the pair reducer through the generic
`reduce_parent_blocks_lanes` machinery erased the win. It measured 83.169 ns
for two parents and 134.35 ns for three parents, so that promotion shape was
rejected. The final production path is exact-only and bypasses the generic
reducer for `out.len() == 2` and `out.len() == 3`.

Post-promotion direct fast-path measurements on `linux-dev`:

| Parents | Production AVX2 | Pair-parent diagnostic | Result vs pre-promotion production |
| ---: | ---: | ---: | --- |
| 1 | 56.131 ns | 54.085 ns | unchanged serial route |
| 2 | 55.948 ns | 54.476 ns | ~20.5% faster |
| 3 | 104.20 ns | 104.54 ns | ~21.5% faster |
| 4 | 102.59 ns | 108.79 ns | ASM preserved |
| 5 | 148.07 ns | 160.22 ns | duplicate-lane route preserved |

Production AVX2 parent-tail ownership is now:

| Parent count | Route |
| ---: | --- |
| 1 | owned serial AVX2 parent reducer |
| 2 | owned two-parent AVX2 reducer |
| 3 | owned two-parent AVX2 reducer plus owned serial parent reducer |
| 4 | `hash_many_avx2` assembly |
| 5/6/7 | owned duplicate-lane AVX2 |
| 8 | owned full AVX2 batch |

No `ct.toml` update is needed. The new branches are selected by public parent
count and CPU capability, and the reducer uses the same key/message inputs as
the existing parent-compression path.

### Ownership Slice: AVX2 Parent Direct-Output Narrowing

Candidate: remove the generic reducer overhead around AVX2 parent reductions by
writing full batches and selected tails directly into the caller's output
buffer. The broad version was rejected. It tried to replace the hosted AVX2
parent reducer with one explicit loop, but exact 5/6/7 and 15-parent rows
regressed. One unsafe bug also proved the value of the raw-CV equivalence test:
the first prototype wrote an 8-lane `hash8_owned` tail directly into a 5/6/7
tail output and Linux aborted with heap corruption. That path was fixed, then
bench-rejected and removed.

Rejected broad-helper measurements on `linux-dev`:

| Parents | Broad direct helper | Previous good route | Decision |
| ---: | ---: | ---: | --- |
| 3 | 106.34 ns | ~104.20 ns | reject |
| 5 | 150.16 ns | ~148.07 ns | reject |
| 6 | 152.47 ns | ~147 ns | reject |
| 7 | 151.21 ns | ~148 ns | reject |
| 15 | 296.61 ns | ~292 ns | reject |

The kept version is narrow:

- exact 2/3 parent reductions keep the direct owned pair-parent routes from the
  previous slice;
- exact 4 parents still use `hash_many_avx2` assembly, but now write directly
  to `out` instead of through the generic reducer temp;
- exact 8 parents use the owned `hash8_owned` full batch directly into `out`;
- all other parent counts keep the previous reducer shape.

Post-narrowing measurements:

| Parents | Production AVX2 | Result |
| ---: | ---: | --- |
| 2 | 54.044 ns | direct owned pair route preserved |
| 3 | 104.11 ns | direct pair-plus-serial route restored |
| 4 | 99.085 ns | small perf win, still assembly |
| 5 | 146.75 ns | restored to owned duplicate-lane route |
| 6 | 146.57 ns | restored to owned duplicate-lane route |
| 7 | 147.41 ns | restored to owned duplicate-lane route |
| 8 | 145.14 ns | owned full batch direct-output route |
| 15 | 292.99 ns | restored to previous reducer shape/no material win |

This slice improves the exact 8-parent owned path and trims overhead from the
remaining exact 4-parent assembly path. It does not change the remaining ASM
ownership surface: parent count 4 still needs a genuinely better owned
schedule or rscrypto-owned assembly. No `ct.toml` update is needed; all branches
are selected only by public parent count and CPU capability.

### Rejected Candidate: SSE4.1 Four-Lane Fallback for AVX2 Count 4

Candidate: route the remaining AVX2 exact 4-chunk and 4-parent tail cases to
the owned SSE4.1 four-lane backend. AVX2 implies SSE4.1, and this would remove
two AVX2 `hash_many_avx2` assembly tail uses if it won.

Result: reject. The owned SSE4.1 backend is correct but slower than the
surviving AVX2 assembly route for both affected count-4 rows on Sapphire
Rapids.

Focused medians on `linux-dev`:

| Row | Owned SSE4.1 | AVX2 production | Decision |
| --- | ---: | ---: | --- |
| `chunk-tail-digest/4` | 1.7259 us | 1.5427 us | keep AVX2 |
| `chunk-tail-cvs/4` | 1.5396 us | 1.4183 us | keep AVX2 |
| `parent-tail-cvs/4` | 108.34 ns | 97.052 ns | keep AVX2 |

No `ct.toml` update is needed. This was benchmark-only forced-kernel
comparison on public kernel labels and public input sizes.

### Diagnostic Guardrail: Full-Batch-Plus-Remainder Counts

The raw CV equivalence test now covers counts:

```text
1..=19, 23, 24, 31, 32
```

This explicitly exercises AVX2 `8 + rem` and AVX-512 `16 + rem` paths, not just
exact sub-degree tails. That matters because the earlier broad AVX2 parent
direct-output prototype had a tail write bug that only shows up in this class
of code. The diagnostic bench matrix now also includes counts `10..14`, so
AVX2 full-batch-plus-remainder behavior is visible without custom benchmark
edits.

Linux `diag_raw_cv_helpers_match_portable` passed with the expanded count set.

Focused AVX2 chunk-CV medians on `linux-dev`:

| Chunks | Production AVX2 | Owned duplicate | Pair diagnostic | Decision |
| ---: | ---: | ---: | ---: | --- |
| 10 | 2.6575 us | 3.3855 us | 4.8593 us | keep production |
| 11 | 3.3663 us | 3.3729 us | 5.9225 us | no material win |
| 12 | 3.1337 us | 3.3728 us | 5.8392 us | keep production |
| 13 | 3.3702 us | 3.3719 us | 6.8952 us | no material win |
| 14 | 3.3857 us | 3.3722 us | 6.8027 us | too small/noisy |

Focused AVX2 parent-CV medians:

| Parents | Production AVX2 | Owned duplicate | Pair diagnostic | Decision |
| ---: | ---: | ---: | ---: | --- |
| 10 | 226.18 ns | 297.81 ns | 270.45 ns | keep production |
| 11 | 286.38 ns | 296.97 ns | 322.42 ns | keep production |
| 12 | 252.06 ns | 296.67 ns | 324.24 ns | keep production |
| 13 | 300.24 ns | 295.81 ns | 376.19 ns | too small/noisy |
| 14 | 298.16 ns | 295.71 ns | 378.04 ns | too small/noisy |

Conclusion: no production promotion from this pass. Counts 13/14 are tempting
on paper, but the edge is ~1-1.5% and the earlier broad reducer already proved
that changing this shape can regress nearby counts. A future attempt needs a
smaller mechanism and a repeat run before promotion.

No `ct.toml` update is needed. This slice only changes diagnostic coverage and
bench count selection; the benchmark branches are public kernel labels and
public input sizes.

### Rejected Candidate: Bounded Parent Block Pointer Helper

Candidate: replace the AVX2 parent direct-output helper's debug-only child
availability proof with a safe bounded slice lookup before passing the child
pair pointer into the ASM ABI. This would make the helper fail closed in release
if a caller violated the parent-count invariant.

Result: reject for now. The safe lookup is correct, and local plus Linux raw-CV
diagnostics passed, but it regressed the exact AVX2 parent hot rows that this
helper was added to improve.

Focused `parent-tail-cvs/rscrypto-x86-avx2` Criterion deltas on `linux-dev`:

| Parents | Median with bounded lookup | Criterion delta | Decision |
| ---: | ---: | ---: | --- |
| 2 | 56.140 ns | +3.8848% | revert |
| 3 | 106.72 ns | +2.5316% | revert |
| 4 | 99.871 ns | +2.9059% | revert |
| 8 | 146.32 ns | +0.6990% | noise, but not enough to save it |

Decision: keep the measured raw pointer helper with the debug assertion. The
callers already prove exact child-pair availability by construction; if this is
revisited, inspect generated assembly and look for a safe form that optimizes
to the same code before promotion.

After reverting, local and Linux `diag_` BLAKE3 tests passed. A longer row-8
rerun with the original helper measured 148.28 ns and Criterion classified it
as within noise, so no production helper change remains from this experiment.

### Safety Hardening: x86 ASM Wrapper Contracts

Tightened the Rust contracts around the remaining BLAKE3 x86 assembly wrappers:
`hash_many_avx2`, `hash_many_avx512`, AVX-512 `xof_many`, and the two AVX-512
`compress_in_place` wrappers now spell out CPU-feature requirements, pointer
validity, output aliasing, ABI narrowing, and public timing boundaries with
numbered `SAFETY` invariants. The AVX-512 XOF callsites now route through the
same local wrapper style instead of calling the external symbol directly.

No production behavior or benchmark route changed. This does not move the
AVX2/AVX-512 assembly files out of the external-derived bucket, but it makes the
remaining vendored surface easier to audit while replacement work continues.

Verification:

```bash
cargo fmt --all
cargo check --features 'blake3,diag,parallel' --bench blake3
cargo test --features 'blake3,diag,parallel' hashes::crypto::blake3::tests::diag_
mutagen sync flush rscrypto-linux
ssh ubuntu@linux-dev 'source ~/.cargo/env && cd /home/ubuntu/rscrypto && cargo bench --profile bench --features "blake3,diag,parallel" --bench blake3 -- "^blake3/xof-output/rscrypto-x86-avx512/4096$" --warm-up-time 1 --measurement-time 3 --sample-size 30'
git diff --check
scripts/check/asm-ledger.sh
```

The Linux XOF-output smoke row measured 1.6996 us for 4096 bytes and Criterion
reported an improvement versus the previous sample, so the wrapper routing did
not introduce a visible AVX-512 XOF regression.

### Ownership Slice: AVX-512 15-Chunk Tail Promoted

Measured the existing owned AVX-512 `hash16_contiguous_owned` diagnostic path
against the production AVX-512 assembly tail. Most sub-degree contiguous chunk
tails are bad fits for the owned full-16 batch because materializing and
duplicating the tail costs more than the assembly cascade. The 15-chunk tail is
the exception: it is nearly a full AVX-512 batch, and the owned 16-way path wins
decisively.

Production now routes only the contiguous 15-chunk AVX-512 tail through the
owned duplicate-lane helper when AVX512DQ is available. This extra guard matters:
the existing assembly AVX-512 kernel intentionally supports `avx512f+avx512vl`
plus AVX2, while the owned Rust intrinsic implementation requires AVX512DQ. CPUs
without DQ keep the assembly path.

Pre-promotion raw chunk-CV medians on `linux-dev`:

| Chunks | AVX-512 ASM tail | Owned duplicate tail | Decision |
| ---: | ---: | ---: | --- |
| 1 | 862.32 ns | 2.7685 us | keep ASM |
| 2 | 870.58 ns | 2.7824 us | keep ASM |
| 3 | 1.7200 us | 2.8587 us | keep ASM |
| 4 | 914.47 ns | 2.8439 us | keep ASM |
| 5 | 1.7999 us | 2.8117 us | keep ASM |
| 6 | 1.8018 us | 2.7881 us | keep ASM |
| 7 | 2.6921 us | 2.8552 us | keep ASM |
| 8 | 1.6191 us | 2.8770 us | keep ASM |
| 15 | 4.3921 us | 2.8110 us | promote owned |

Post-promotion focused medians:

| Row | Median | Result |
| --- | ---: | --- |
| `blake3/chunk-tail-cvs/rscrypto-x86-avx512/15` | 2.7352 us | -36.989% vs old production row |
| `blake3/chunk-tail-cvs/rscrypto-x86-avx512-owned-hash-many/15` | 2.7376 us | diagnostic matches promoted route |
| `blake3/chunk-tail-digest/rscrypto/15` | 3.4729 us | public row stays well ahead of upstream |
| `blake3/chunk-tail-digest/rscrypto-x86-avx512/15` | 3.4570 us | forced AVX-512 row confirms path |
| `blake3/chunk-tail-digest/blake3/15` | 4.6885 us | upstream comparison |

Verification:

```bash
cargo fmt --all
cargo check --features 'blake3,diag,parallel' --bench blake3
cargo test --features 'blake3,diag,parallel' diag_
git diff --check
scripts/check/asm-ledger.sh
mutagen sync flush rscrypto-linux
ssh ubuntu@linux-dev 'source ~/.cargo/env && cd /home/ubuntu/rscrypto && cargo check --features "blake3,diag,parallel" --bench blake3'
ssh ubuntu@linux-dev 'source ~/.cargo/env && cd /home/ubuntu/rscrypto && cargo test --features "blake3,diag,parallel" diag_'
ssh ubuntu@linux-dev 'source ~/.cargo/env && cd /home/ubuntu/rscrypto && cargo bench --profile bench --features "blake3,diag,parallel" --bench blake3 -- "^blake3/chunk-tail-cvs/rscrypto-x86-avx512.*" --warm-up-time 1 --measurement-time 2 --sample-size 20'
ssh ubuntu@linux-dev 'source ~/.cargo/env && cd /home/ubuntu/rscrypto && cargo bench --profile bench --features "blake3,diag,parallel" --bench blake3 -- "(^blake3/chunk-tail-cvs/rscrypto-x86-avx512.*/15$|^blake3/chunk-tail-digest/(rscrypto|rscrypto-x86-avx512|blake3)/15$)" --warm-up-time 1 --measurement-time 3 --sample-size 30'
```

No `ct.toml` update is needed. The branch is selected by public chunk count and
public CPU capability. It does not branch on key material, message contents, or
secret-derived state.

### Ownership Slice: AVX-512 15-Parent Tail Promoted

Split the owned AVX-512 hash-many primitive into a generic 16-lane
`hash16_owned` helper plus the existing contiguous chunk wrapper. This gives
diagnostics a real owned AVX-512 parent-reduction candidate instead of
measuring only chunk-specialized work.

The parent sweep matched the AVX-512 chunk-tail pattern: small parent counts are
still better on assembly, but the 15-parent tail is nearly a full 16-lane batch
and wins through the owned path. Production now routes only `rem == 15` parent
tails through the owned helper, behind the same AVX512DQ runtime guard as the
15-chunk tail. Full AVX-512 batches and smaller tails stay on assembly.

Pre-promotion raw parent-CV medians on `linux-dev`:

| Parents | AVX-512 ASM tail | Owned 16-lane helper | Decision |
| ---: | ---: | ---: | --- |
| 1 | 56.876 ns | 196.04 ns | keep ASM |
| 2 | 60.036 ns | 209.56 ns | keep ASM |
| 3 | 108.24 ns | 188.43 ns | keep ASM |
| 4 | 72.809 ns | 192.59 ns | keep ASM |
| 5 | 129.19 ns | 206.72 ns | keep ASM |
| 6 | 132.77 ns | 185.05 ns | keep ASM |
| 7 | 176.73 ns | 189.03 ns | keep ASM |
| 8 | 126.04 ns | 201.34 ns | keep ASM |
| 15 | 303.66 ns | 199.43 ns | promote owned |

Post-promotion focused medians:

| Row | Median | Result |
| --- | ---: | --- |
| `blake3/parent-tail-cvs/rscrypto-x86-avx512/15` | 194.91 ns | -34.302% vs old production row |
| `blake3/parent-tail-cvs/rscrypto-x86-avx512-owned-hash-many/15` | 200.12 ns | diagnostic matches promoted route |

Verification:

```bash
cargo fmt --all
cargo check --features 'blake3,diag,parallel' --bench blake3
cargo test --features 'blake3,diag,parallel' diag_
mutagen sync flush rscrypto-linux
ssh ubuntu@linux-dev 'source ~/.cargo/env && cd /home/ubuntu/rscrypto && cargo check --features "blake3,diag,parallel" --bench blake3 && cargo test --features "blake3,diag,parallel" diag_'
ssh ubuntu@linux-dev 'source ~/.cargo/env && cd /home/ubuntu/rscrypto && cargo bench --profile bench --features "blake3,diag,parallel" --bench blake3 -- "^blake3/parent-tail-cvs/rscrypto-x86-avx512.*" --warm-up-time 1 --measurement-time 2 --sample-size 20'
ssh ubuntu@linux-dev 'source ~/.cargo/env && cd /home/ubuntu/rscrypto && cargo bench --profile bench --features "blake3,diag,parallel" --bench blake3 -- "^blake3/parent-tail-cvs/rscrypto-x86-avx512.*/15$" --warm-up-time 1 --measurement-time 3 --sample-size 30'
```

No `ct.toml` update is needed. The branch is selected by public parent count
and public CPU capability. It does not branch on key material, message
contents, or secret-derived state.

### Rejected Boundary: AVX-512 Full 16-Lane Hash-Many Promotion

Added `16` to the BLAKE3 tail diagnostic count set so the same benchmark group
also covers the AVX-512 full-degree boundary. This checks whether the owned
16-lane helper should replace the remaining full-batch AVX-512 `hash_many`
assembly path for contiguous chunk CVs or parent CV reductions.

Result: reject. The owned helper is correct but slower than the surviving
assembly entrypoint for both full 16-chunk hashing and full 16-parent
reduction.

Linux Sapphire Rapids medians:

| Row | ASM/production median | Owned diagnostic median | Decision |
| --- | ---: | ---: | --- |
| `blake3/chunk-tail-cvs/.../16` | 2.4103 us | 2.6433 us | keep asm |
| `blake3/parent-tail-cvs/.../16` | 180.12 ns | 196.85 ns | keep asm |

Command:

```bash
mutagen sync flush rscrypto-linux
ssh ubuntu@linux-dev 'source ~/.cargo/env && cd /home/ubuntu/rscrypto && cargo check --features "blake3,diag,parallel" --bench blake3 && cargo bench --profile bench --features "blake3,diag,parallel" --bench blake3 -- "^blake3/(chunk-tail-cvs|parent-tail-cvs)/rscrypto-x86-avx512.*/16$" --warm-up-time 1 --measurement-time 3 --sample-size 30'
```

No `ct.toml` update is needed. This was a benchmark-only diagnostic expansion;
no production branch or secret boundary changed.

### Rejected Candidate: AVX-512 Exact-Block Owned Hash-Many

The `x86-avx512-owned-hash-many` diagnostic originally measured owned
contiguous full-chunk batches and sub-degree chunk tails, but exact-block
one-chunk inputs still used the direct AVX-512 `hash_many` assembly shortcut.
That made 64/256/1024 byte forced rows misleading for this candidate.

Added a diagnostic-only `owned_x86_hash_many` marker and routed AVX-512
exact-block one-chunk digest/keyed/XOF prefixes through
`avx512::hash16_owned` with duplicated input lanes. Production routing is
unchanged; this only makes the forced diagnostic label honest.

Result: reject hard. The duplicate-lane owned AVX-512 helper is effectively tied
for 64-byte rows, but 256-byte and 1024-byte exact-block rows are roughly 2.5-3x
slower than current production routing.

Linux Sapphire Rapids medians:

| Row | Current AVX-512 | Owned exact-block hash-many | Upstream `blake3` | Decision |
| --- | ---: | ---: | ---: | --- |
| `blake3/64` | 53.790 ns | 53.826 ns | 54.902 ns | tie; no win |
| `blake3/256` | 216.56 ns | 660.49 ns | 250.59 ns | reject |
| `blake3/1024` | 864.62 ns | 2.5856 us | 918.03 ns | reject |
| `blake3/keyed/64` | 52.932 ns | 52.719 ns | 54.463 ns | tie; no win |
| `blake3/keyed/256` | 220.40 ns | 682.41 ns | 227.81 ns | reject |
| `blake3/keyed/1024` | 917.01 ns | 2.7709 us | 914.67 ns | reject |
| `blake3/xof/64` | 58.757 ns | 58.396 ns | 58.433 ns | tie; no win |
| `blake3/xof/256` | 226.38 ns | 559.02 ns | 227.35 ns | reject |
| `blake3/xof/1024` | 887.33 ns | 2.4573 us | 899.22 ns | reject |

Command:

```bash
mutagen sync flush rscrypto-linux
ssh ubuntu@linux-dev 'source ~/.cargo/env && cd /home/ubuntu/rscrypto && cargo check --features "blake3,diag,parallel" --bench blake3 && cargo test --features "blake3,diag,parallel" hashes::crypto::blake3::tests::diag_forced_kernels_match_normal_paths && cargo bench --profile bench --features "blake3,diag,parallel" --bench blake3 -- "^(blake3|blake3/keyed|blake3/xof)/(rscrypto|rscrypto-x86-avx512|rscrypto-x86-avx512-owned-hash-many|blake3)/(64|256|1024)$" --warm-up-time 1 --measurement-time 3 --sample-size 30'
```

Conclusion: keep the diagnostic path because it closes a measurement blind spot,
but do not promote this mechanism. The remaining exact-block work needs a
dedicated one-lane/four-block owned schedule or an rscrypto-owned assembly
entrypoint that removes real setup/ABI cost. Duplicating work into all 16 lanes
is the wrong shape.

No `ct.toml` update is needed. The new route is diagnostic-only and selected by
public kernel label, public input length, and public CPU capability.

### Diagnostic Slice: AVX-512 Exact-Block ASM Selector

Added `x86-avx512-exact-block-asm` as a diagnostic-only kernel label. It uses
the normal AVX-512 kernel but disables the narrow 256B AVX2 retarget inside
`digest_one_chunk_root_hash_words_x86`, so the benchmark can distinguish
"forced AVX-512 selected" from "forced AVX-512 but retargeted to AVX2 for the
exact four-block case."

Result: no production promotion on Sapphire Rapids. This host is AMX-capable,
so `allow_avx2_hash_many_one_chunk_fast_path()` is false and normal forced
AVX-512 was already using the direct AVX-512 assembly exact-block path. The new
label is still useful because it closes the diagnostic ambiguity and will let
Ice Lake/Zen checks prove whether the AVX2 retarget branch is helping or
hurting when those hosts are reachable.

Linux Sapphire Rapids 256B medians:

| Row | Public rscrypto | Forced AVX2 | Forced AVX-512 | Forced AVX-512 exact asm | Upstream `blake3` | Decision |
| --- | ---: | ---: | ---: | ---: | ---: | --- |
| `blake3/256` | 216.87 ns | 253.11 ns | 218.57 ns | 218.31 ns | 252.41 ns | keep current direct AVX-512 asm |
| `blake3/keyed/256` | 216.78 ns | 252.84 ns | 218.04 ns | 217.66 ns | 229.47 ns | keep current direct AVX-512 asm |
| `blake3/xof/256` | 225.60 ns | 282.52 ns | 228.97 ns | 227.93 ns | 231.69 ns | no action; XOF already used AVX-512 asm |

Command:

```bash
mutagen sync flush rscrypto-linux
ssh ubuntu@linux-dev 'source ~/.cargo/env && cd /home/ubuntu/rscrypto && cargo check --features "blake3,diag,parallel" --bench blake3 && cargo test --features "blake3,diag,parallel" hashes::crypto::blake3::tests::diag_forced_kernels_match_normal_paths && cargo bench --profile bench --features "blake3,diag,parallel" --bench blake3 -- "^(blake3|blake3/keyed|blake3/xof)/(rscrypto|rscrypto-x86-avx2|rscrypto-x86-avx512|rscrypto-x86-avx512-exact-block-asm|blake3)/(64|256|1024)$" --warm-up-time 1 --measurement-time 3 --sample-size 30'
```

No `ct.toml` update is needed. The selector is diagnostic-only and branches on
the public benchmark kernel label, public input length, and public CPU
capability.

### Rejected Candidate: AVX-512 Owned XOF Output Cascade

Added a bulk XOF-output benchmark so root-output emitter changes are visible:

- `blake3/xof-output`

The existing `blake3/xof` bench squeezes only 64 bytes and mostly measures
hashing/finalization. The new group fixes the input at 4096 bytes and varies
the requested XOF output length, so the AVX-512 `xof_many` path can be judged
directly.

Candidate: enable the existing owned AVX-512 `root_output_blocks16` intrinsic
body on supported OSes and remove the supported-OS dispatcher shortcut that
sent all AVX-512 bulk output reads to `rscrypto_blake3_xof_many_avx512`
assembly. This would have used the owned 16/8/4/2/1 cascade already present in
the fallback path.

Baseline raw medians on `linux-dev` before the candidate:

| Output bytes | Public rscrypto | Forced AVX-512 | Upstream `blake3` |
| ---: | ---: | ---: | ---: |
| 64 | 1.0631 us | 1.0972 us | 1.0987 us |
| 128 | 1.0974 us | 1.0913 us | 1.0817 us |
| 256 | 1.0748 us | 1.1246 us | 1.1532 us |
| 512 | 1.1594 us | 1.1971 us | 1.1865 us |
| 1024 | 1.1676 us | 1.2085 us | 1.1816 us |
| 4096 | 1.6700 us | 1.6986 us | 1.7095 us |

Candidate medians:

| Output bytes | Public rscrypto | Public change | Forced AVX-512 | Forced change | Decision |
| ---: | ---: | ---: | ---: | ---: | --- |
| 64 | 1.0644 us | no change | 1.0517 us | -5.3033% | not enough |
| 128 | 1.1157 us | noise | 1.1408 us | +5.2515% | regress |
| 256 | 1.1596 us | +8.8306% | 1.1677 us | noise | regress |
| 512 | 1.1503 us | noise | 1.1694 us | -3.3367% | mixed |
| 1024 | 1.2391 us | +6.6167% | 1.2109 us | no change | regress |
| 4096 | 1.7358 us | +4.0213% | 1.7980 us | noise | regress |

The candidate was reverted. A forced 64-byte win is not worth broad public
regressions at 256, 1024, and 4096 bytes. The remaining AVX-512 XOF assembly
entrypoint stays live until we have a narrower owned emitter that wins the
public rows, not just one forced diagnostic row.

Verification:

```bash
cargo fmt --all
cargo check --features 'blake3,diag,parallel' --bench blake3
cargo test --features 'blake3,diag,parallel' xof_
cargo test --features 'blake3,diag,parallel' hashes::crypto::blake3::tests::diag_forced_kernels_match_normal_paths
mutagen sync flush rscrypto-linux
ssh ubuntu@linux-dev 'source ~/.cargo/env && cd /home/ubuntu/rscrypto && cargo check --features "blake3,diag,parallel" --bench blake3 && cargo test --features "blake3,diag,parallel" xof_ && cargo test --features "blake3,diag,parallel" hashes::crypto::blake3::tests::diag_forced_kernels_match_normal_paths'
ssh ubuntu@linux-dev 'source ~/.cargo/env && cd /home/ubuntu/rscrypto && cargo bench --features "blake3,diag,parallel" --bench blake3 -- "^blake3/xof-output/(rscrypto|rscrypto-x86-avx512|blake3)/(64|128|256|512|1024|4096)$" --warm-up-time 1 --measurement-time 2 --sample-size 20'
```

No `ct.toml` update is needed. The candidate changed only public output length
dispatch and was reverted; the new benchmark does not change production
behavior.

### Cross-Arch Check: Intel Ice Lake 256B Exact-Block Gap Narrowed

The AVX2 exact-block one-chunk promotion was originally verified on Sapphire
Rapids. Ice Lake was one of the original 256B loss platforms, so this run checks
whether the promotion generalizes.

Host: `linux-icl`, Intel Xeon Platinum 8375C, Ice Lake, 4 vCPU KVM guest.

Command:

```bash
mutagen sync flush rscrypto-intel-icl
ssh ubuntu@linux-icl 'bash -lc "cd /home/ubuntu/rscrypto && cargo check --features \"blake3,diag,parallel\" --bench blake3 && cargo test --features \"blake3,diag,parallel\" hashes::crypto::blake3::tests::diag_forced_kernels_match_normal_paths && cargo bench --features \"blake3,diag,parallel\" --bench blake3 -- \"^(blake3|blake3/keyed|blake3/xof)/(rscrypto|rscrypto-x86-avx2|rscrypto-x86-avx512|blake3)/(64|256|1024)$\" --warm-up-time 2 --measurement-time 4 --sample-size 30"'
```

Correctness passed. Focused median results:

| Row | Public rscrypto | Forced AVX2 | Forced AVX-512 | Upstream | Result |
| --- | ---: | ---: | ---: | ---: | --- |
| `blake3/64` | 52.768 ns | 57.987 ns | 52.579 ns | 53.232 ns | public ahead |
| `blake3/256` | 243.49 ns | 244.88 ns | 246.04 ns | 230.59 ns | gap narrowed, still behind |
| `blake3/1024` | 811.66 ns | 998.05 ns | 812.73 ns | 854.34 ns | public ahead |
| `blake3/keyed/64` | 52.531 ns | 58.668 ns | 53.193 ns | 54.037 ns | public ahead |
| `blake3/keyed/256` | 242.56 ns | 245.63 ns | 246.30 ns | 213.41 ns | gap narrowed, still behind |
| `blake3/keyed/1024` | 813.36 ns | 998.73 ns | 812.96 ns | 837.19 ns | public ahead |
| `blake3/xof/64` | 53.915 ns | 59.643 ns | 52.032 ns | 59.892 ns | public ahead |
| `blake3/xof/256` | 219.41 ns | 252.92 ns | 214.54 ns | 213.77 ns | tie/slightly behind |
| `blake3/xof/1024` | 825.46 ns | 1.0076 us | 820.39 ns | 837.58 ns | public ahead |

Compared with the 2026-06-22 baseline, `oneshot/256` moved from 255.39 ns to
243.49 ns and `keyed/256` moved from 255.57 ns to 242.56 ns on Ice Lake. That
is real progress, but not enough: upstream remains ~5.6% faster for oneshot
256B and ~13.7% faster for keyed 256B.

Conclusion: keep the AVX2 exact-block promotion, but do not call the 256B
Ice Lake gap closed. The next 256B implementation target, if we keep pursuing
that point, has to remove real setup/ABI cost rather than reshuffle the same
four compression calls. Short XOF no longer looks like the priority on Ice Lake.

No `ct.toml` update is needed. This was verification of existing public paths
and forced diagnostic paths; no new secret boundary was added.

### Rejected Slice: AVX2 Exact-Four Straight-Line Chain

Candidate: replace the exact-4-block case in `avx2_owned_exact_block_chain`
with a dedicated helper that calls `compress_cv_avx2_bytes` four times
straight-line, avoiding the runtime loop and branch checks for 256B inputs.

Decision: reject and revert. The helper was byte-correct, but slower on the
Sapphire Rapids Linux box. The loop is not the relevant bottleneck there.

Command:

```bash
mutagen sync flush rscrypto-linux
ssh ubuntu@linux-dev 'source ~/.cargo/env && cd /home/ubuntu/rscrypto && cargo check --features "blake3,diag,parallel" --bench blake3 && cargo test --features "blake3,diag,parallel" hashes::crypto::blake3::tests::diag_forced_kernels_match_normal_paths && cargo bench --features "blake3,diag,parallel" --bench blake3 -- "^(blake3|blake3/keyed)/(rscrypto|rscrypto-x86-avx2|rscrypto-x86-avx512|blake3)/256$" --warm-up-time 2 --measurement-time 4 --sample-size 30'
```

Correctness passed before benchmarking. Focused median results from the
rejected build:

| Row | Median | Criterion change | Decision |
| --- | ---: | ---: | --- |
| `blake3/rscrypto/256` | 218.87 ns | +1.9388% | reject |
| `blake3/rscrypto-x86-avx2/256` | 251.76 ns | +1.3949% | reject |
| `blake3/rscrypto-x86-avx512/256` | 217.69 ns | +0.2348% | noise |
| `blake3/blake3/256` | 254.09 ns | +0.0219% | unchanged |
| `blake3/keyed/rscrypto/256` | 219.58 ns | +1.8800% | reject |
| `blake3/keyed/rscrypto-x86-avx2/256` | 257.48 ns | +3.1232% | reject |
| `blake3/keyed/rscrypto-x86-avx512/256` | 221.00 ns | -0.6553% | noise |
| `blake3/keyed/blake3/256` | 233.99 ns | -0.0386% | unchanged |

No `ct.toml` update is needed. The rejected helper changed only public-length
dispatch shape and did not survive.

### Current Check: Sapphire Streaming and Short XOF Gaps Are Stale

The 2026-06-22 matrix still lists Sapphire Rapids streaming and short-XOF rows
as losses, but the current tree has moved since that run. Re-measured the active
Linux Sapphire Rapids box before spending more code on these paths.

Host: `linux-dev`, Intel Xeon Platinum 8488C/Sapphire Rapids class, AVX-512
enabled.

Streaming command:

```bash
mutagen sync flush rscrypto-linux
ssh ubuntu@linux-dev 'source ~/.cargo/env && cd /home/ubuntu/rscrypto && cargo bench --features "blake3,diag,parallel" --bench blake3 -- "^blake3/streaming/(rscrypto|rscrypto-x86-avx2|rscrypto-x86-avx512|blake3)/(64|4096|16384|65536)B$" --warm-up-time 2 --measurement-time 4 --sample-size 30'
```

Current streaming medians:

| Row | Public rscrypto | Forced AVX2 | Forced AVX-512 | Upstream | Result |
| --- | ---: | ---: | ---: | ---: | --- |
| `streaming/64B` | 1.0118 ms | 1.1784 ms | 1.0049 ms | 1.0481 ms | public ahead |
| `streaming/4096B` | 285.84 us | 419.32 us | 285.15 us | 284.73 us | tie |
| `streaming/16384B` | 179.67 us | 246.56 us | 177.76 us | 180.00 us | tie/ahead |
| `streaming/65536B` | 170.17 us | 234.44 us | 166.89 us | 167.25 us | tie |

Short-XOF command:

```bash
ssh ubuntu@linux-dev 'source ~/.cargo/env && cd /home/ubuntu/rscrypto && cargo bench --features "blake3,diag,parallel" --bench blake3 -- "^blake3/xof/(rscrypto|rscrypto-x86-avx2|rscrypto-x86-avx512|blake3)/(0|1|32|64)$" --warm-up-time 2 --measurement-time 4 --sample-size 30'
```

Current short-XOF medians:

| Row | Public rscrypto | Forced AVX2 | Forced AVX-512 | Upstream | Result |
| --- | ---: | ---: | ---: | ---: | --- |
| `xof/0` | 52.140 ns | 62.279 ns | 51.762 ns | 51.925 ns | tie |
| `xof/1` | 77.070 ns | 85.151 ns | 78.138 ns | 78.440 ns | tie/ahead |
| `xof/32` | 53.192 ns | 62.784 ns | 53.647 ns | 54.963 ns | public ahead |
| `xof/64` | 58.953 ns | 72.280 ns | 58.318 ns | 59.260 ns | tie/ahead |

Decision: do not spend the next ownership slice on Sapphire streaming or short
XOF. The current public route already uses AVX-512 where it should, and forced
AVX2 is clearly worse for these rows. Remaining streaming/XOF questions are
cross-arch verification questions, especially AMD Zen5, not Sapphire code
targets.

Zen5 note: `amd-zen5` exists in the runner catalog as `linux-zen5`, but no Zen5
instance is currently provisioned and the AWS budget tripwire is already over
limit. Do not start a new Zen5 instance without explicit approval.

No `ct.toml` update is needed. This was benchmark-only verification of public
and forced diagnostic paths.

## Ownership Boundary

The BLAKE3 x86_64 files remain external-derived until every surviving assembly
entrypoint is rewritten or regenerated from rscrypto-owned source:

- `src/hashes/crypto/blake3/x86_64/asm/rscrypto_blake3_avx2_x86-64_*.s`
- `src/hashes/crypto/blake3/x86_64/asm/rscrypto_blake3_avx512_x86-64_*.s`

Do not bulk-port another project. New assembly should match rscrypto's ABI and
dispatch needs directly, with small entrypoints where the current generic
upstream ABI costs us cycles.

## Future BLAKE3 Work

Phase 1 is closed. Future BLAKE3 work should start only when a new owned
AVX2/AVX-512 design has a plausible route to beating the retained
upstream-derived assembly.

1. Maintain the gap matrix.
   Verify with `python3 scripts/bench/blake3-gap-report.py --root benchmark_results/2026-06-22/linux --top 18`.

2. Keep forced-backend diagnostics available.
   Verify current ASM, Rust intrinsics, and any future owned entrypoint can be
   benchmarked independently on the same op/size rows.

3. Re-check AMD Zen5 before changing 256B policy.
   AVX2/Sapphire Rapids and Intel Ice Lake have been measured; AMD Zen5 still
   needs verification. Do not pursue a plain exact-four unroll or AVX-512
   duplicate-lane exact-block `hash_many`; both were measured and rejected.

4. Finish measured sub-degree tails.
   Contiguous 1/2/3/5/6/7 chunk tails are now owned through the measured AVX2
   routes. Contiguous 4-chunk tails still use `hash_many_avx2` assembly.
   Exact 1/2/3/5/6/7/8 parent reductions are owned through measured AVX2
   routes, but 2/3-parent remainders after earlier full batches still use
   `hash_many_avx2` assembly because the broad direct reducer regressed. Exact
   4-parent reductions also still use `hash_many_avx2` assembly, though the
   direct-output micro-route trims generic reducer overhead. Do not use serial
   AVX2 beyond the measured one-chunk and one-parent tails, SSE4.1 lower-width
   fallback tails, or a VEX-coded clone of the SSE4.1 4-way shape; those broader
   routes were measured and rejected on Sapphire Rapids. A duplicate-lane route
   is acceptable only for the measured winning counts unless another per-degree
   benchmark proves otherwise.

   AVX-512 contiguous chunk tails are split more narrowly: only 15 chunks is
   owned today, behind an AVX512DQ runtime guard. Tails 1-8 were measured slower
   through the owned full-16 duplicate-lane route and must stay on assembly
   unless a real masked/narrow owned kernel beats them.

   AVX-512 parent tails follow the same rule: only the 15-parent tail is owned
   today, behind the AVX512DQ guard. Parent tails 1-8 measured slower through
   the owned 16-lane helper and must stay on assembly unless a different owned
   parent kernel beats them. The full 16-parent boundary was also measured
   slower through the owned helper, so the current full-batch parent reduction
   remains assembly-backed too.

5. Verify remaining short-XOF gaps cross-arch before writing code.
   Sapphire Rapids now ties or beats upstream for `xof/0`, `xof/1`, `xof/32`,
   and `xof/64`. The broad AVX-512 owned XOF-output cascade was measured and
   reverted; keep `xof_many` assembly until a narrower owned emitter wins the
   `blake3/xof-output` public rows. Re-check AMD Zen5 before spending code on
   this path.

6. Verify streaming policy cross-arch before writing more policy code.
   Sapphire Rapids now ties or beats upstream for `64B`, `4096B`, `16384B`, and
   `65536B` streaming update sizes. Re-check AMD Zen5 and Intel Ice Lake before
   changing the policy again.

7. Replace the remaining AVX2/AVX-512 `hash_many` and `compress_in_place`
   assembly.
   Verify backend equivalence tests cover AVX2, AVX-512, unaligned input, all
   tail sizes, parent nodes, keyed mode, derive-key mode, and XOF.

8. Move the ledger classification only after the replacement is real.
   Verify `scripts/check/asm-ledger.sh` reports the BLAKE3 x86_64 files as
   rscrypto-owned only when the upstream-derived files are gone or fully
   rewritten.

## Current Hypotheses

- The remaining `256` byte keyed/oneshot losses are not fixed by a plain
  exact-four unroll. If this row is attacked again, remove real setup/ABI cost
  or add a genuinely better owned entrypoint.
- AVX-512 duplicate-lane exact-block `hash_many` is not a viable owned
  replacement. It ties 64B but regresses exact-block 256B/1024B digest, keyed,
  and XOF rows badly.
- On Sapphire Rapids, forced AVX-512 256B digest/keyed already means direct
  AVX-512 asm, not the AVX2 retarget. The diagnostic
  `x86-avx512-exact-block-asm` label exists to make that distinction visible on
  hosts where the retarget branch is enabled.
- Sapphire short-XOF is no longer a live loss in the current tree. Re-check AMD
  Zen5 before spending code on XOF root/finalization. The broad AVX-512 owned
  XOF-output cascade regressed public 256/1024/4096-byte output rows and should
  not be repeated without a narrower mechanism.
- Sapphire streaming is no longer a live loss in the current tree. Re-check AMD
  Zen5 and Intel Ice Lake before changing update/bulk policy again.
- AVX2 contiguous chunk tails are split: 1/2/3/5/6/7 are owned and faster on
  Sapphire Rapids; 4 still belongs to the assembly tail until a genuinely
  tail-specific owned schedule beats it.
- AVX2 partial parent tails are split: exact 1/2/3/5/6/7/8 reductions are owned
  and faster on Sapphire Rapids, while exact 4 and generic 2/3/4 remainders
  remain assembly-backed. The rejected broad serial, broad direct reducer, and
  SSE4.1 fallback experiments should not be repeated without a new mechanism.
- AVX-512 full 16-lane `hash_many` replacement is not currently a win: owned
  diagnostics are slower for both full 16-chunk CV hashing and full 16-parent
  reduction on Sapphire Rapids.
- `derive-key` is already strong on x86_64. Keep it as a regression guard, not
  the first tuning target.

## Constant-Time Notes

Existing `ct.toml` coverage includes BLAKE3 keyed verification, BLAKE3 keyed
fixed-vs-random key hashing, and parallel keyed hashing. No CT manifest change
is needed before the first performance diagnostics.

Update `ct.toml` when a new owned ASM entrypoint changes the secret boundary,
adds a key-dependent fast path, or bypasses an existing harnessed path. Candidate
future cases:

- exact 256B keyed digest with fixed vs random key;
- parallel keyed digest forced through the new owned `hash_many` entrypoint;
- keyed XOF/root output if a dedicated keyed XOF assembly path is added.