Skip to main content

Crate tachyon

Crate tachyon 

Source
Expand description

Measure how fast this host’s memory subsystem is right now.

§Why this exists

Benchmark wall times on shared cloud hosts are not comparable across runs. Measured on this project’s own fleet (2026-08-06, wgs-5M/m7i, one binary, one input): repeating a replicate on the SAME EC2 instance reproduces within 2.03%, while letting each replicate pick a fresh instance spreads 9.40% — and on one sample the same unchanged binary measured 18.89 s and 25.01 s depending only on which machine it landed on.

That variance is not lost CPU. Across every replicate cpu_time / wall held at 14.1–14.7 of 16 vCPUs, so the scheduler was giving us our cores; what changed was that identical work burned ~20% more CPU-seconds. Cores were stalling on memory, because co-tenants on the same socket were consuming last-level cache and DRAM bandwidth that the instance contract never promised us.

So the probe measures memory access rate under whatever contention exists at this moment, and nothing else.

§Why a pointer chase, and why several threads

The probe has to be sensitive to the same resource the workload is sensitive to. bwa-mem’s FMI search is a dependent chain of random accesses over a multi-gigabyte index — latency- and memory-parallelism-bound, not ALU-bound. A tight arithmetic loop in registers would read the same on a contended host as on an idle one and tell you nothing.

A pointer chase reproduces that shape: each step’s address comes from the previous step’s value, so the CPU cannot prefetch and cannot overlap the misses within a thread. Running one chain per thread then reproduces the aggregate memory parallelism of a -t 16 alignment, which is what a noisy neighbour actually degrades.

§Why it must not resemble the code under test

The probe is deliberately independent of bwa-mem3. If it shared code, then dividing a measurement by a probe score would cancel real regressions along with host noise. cpu_time is the cautionary example: it is already a contention signal, but a genuinely slower binary also burns more of it, so normalising by it would flatten everything including the thing you are looking for. A separate pointer chase cannot move when bwa-mem3 changes, and does move when the host degrades — which is exactly the property that makes it usable as a control.

§Does it actually respond to a neighbour?

Yes, measured. On an otherwise quiet 12-core host, a 4-thread probe against the same 4-thread probe with an 8-thread memory neighbour beside it:

  alone                    28.4 / 30.3 / 31.7 M accesses/s   126-141 ns
  8-thread neighbour       18.0 / 17.2 / 17.9 M accesses/s   222-233 ns
  alone again              29.8 / 29.7 M accesses/s          134 ns

A 42% drop, tight within each condition (4.7% spread contended), and it recovers when the neighbour leaves — so the score tracks conditions now rather than drifting. The response is larger than the ~20% wall-time effect it is meant to detect, which is the right way round for a detector.

Two caveats that shape how it should be read. It is not a pure memory probe: 8 spinning CPU-only threads with no memory traffic still cost ~18%, so the score conflates memory contention with CPU availability. For deciding “is this host degraded” that is harmless — either cause means it is — but it matters if the score is ever used as a divisor. And the probe needs a quiet baseline to calibrate against: an early attempt here read 12-22 M/s purely because a build was finishing in the background, which inverted the result until the machine was actually idle.

§Filter first, normalise only once validated

Treating the score as a scale factor assumes probe and workload degrade proportionally. That is plausible and unverified. Treating it as a filter only assumes a bad score means a bad host, which is far weaker. Collect scores next to timings, join on the recorded instance id, and check how much of the between-host variance the score explains before dividing by it.

Structs§

ProbeConfig
How the probe should be run.
ProbeResult
What the probe measured.

Functions§

run
Run the probe.
slots_for
Number of chain slots for a given working set, rounded down to whole cache lines. The resulting array is exactly working_set_bytes when the argument is a whole number of cache lines.