# bench
Two harnesses, measuring different things. Neither is a substitute for the other, and
knowing which one answers your question is most of the value here.
| `bench/throughput.sh` | end-to-end req/s and req/CPU-second against nginx | when the machine differs from the one a quoted number came from |
| `cargo bench` (criterion) | per-call cost of `handle_request` and path resolution | when the change you made is *outside* those two functions |
## Throughput, against nginx
```bash
./bench/throughput.sh # both servers
SERVER_ONLY=1 ./bench/throughput.sh # skip nginx
DURATION=30s CONNECTIONS=100 ./bench/throughput.sh
```
Needs `oha` (`cargo install oha`) and, for the comparison, `nginx` on `PATH`. It prints
the host, core count, nginx version and crate version alongside the result, so any figure
copied out of it can be checked against the machine that produced it.
**The metric is req/CPU-second, not req/s.** Throughput can be bought with cores;
efficiency cannot. Both are printed, and the CPU column should read ~100% for a
single-worker server under saturating load — if it does not, the run was not saturating
and the numbers mean nothing.
### Fairness rules, all load-bearing
- **One worker each.** The server example uses `#[tokio::main(flavor = "current_thread")]`;
nginx is configured `worker_processes 1`. A comparison between a multi-threaded server
and a single-worker nginx measures the machine, not the software.
- **Identical files**, served from the same directory, generated by the script so the two
cannot drift apart.
- **A plain server.** `examples/bench_server.rs` enables nothing — no SPA mode, no live
reload, no immutable-asset predicate. Each of those adds per-request work, and a
baseline should measure the floor rather than one deployment's feature set.
- **Keep-alive on both sides**, which is the default for HTTP/1.1 and for `oha`. A server
that closed per request would look 4x slower for a reason unrelated to its file serving.
- **CPU is read from the server process's own accounting** (`ps -o cputime`), sampled
either side of the measured window, so the load generator's CPU is not counted against
the server. For nginx that is the *worker* pid; the master only supervises.
### Three ways this harness was wrong before it was right
Recorded because each produced a confident, plausible, wrong number:
1. **No warm-up: the first measured run read 14,704 req/s where every subsequent run read
60,475.** Cold binary pages and a cold page cache. Recording that first number would
have reported a catastrophic regression that did not exist. There is now a discarded
3-second warm-up before the measured window.
2. **`oha` reports `Total: 3000.7195 ms`, and the script read it as seconds.** A clean
1000x error, which made CPU% round to zero and req/CPU-second read as 74 *million*.
The unit is now parsed.
3. **The original 56,700-vs-52,500 figures named no machine.** They were quoted in
`README.md` as a flat competitive claim for months. Re-run on a Mac16,2 the same crate
serves 60,877 req/s while nginx reaches 74,566 — the claim inverts, not because the
crate slowed but because nginx scales better on that hardware. An unreproducible
benchmark is not evidence, and this is what `mini-serve`'s bench README means by
"record the date and hardware with any figure quoted elsewhere, since both move."
## Criterion micro-benchmarks
```bash
cargo bench # both suites
cargo bench --bench handle_request -- --save-baseline before
# ... make a change ...
cargo bench --bench handle_request -- --baseline before
```
`handle_request` covers `small_file_200`, `large_file_200`, `not_modified_304` and
`sidecar_hit_200`. `path_resolution` covers shallow, deep, percent-decoded, traversal-
rejected and directory-index paths.
**Read the direction carefully.** `--baseline before` reports how the *current* code
compares to the saved baseline: "Performance has regressed" means the run you just did is
slower than the baseline, whichever code that was. Comparing an old checkout against a
baseline saved from new code inverts every sign, which is easy to do and easy to
misreport.
### What they cannot see
These measure two functions. They do not measure the accept loop, connection handling,
response writing, or anything a caller wraps around them — so a change to those is
invisible here and needs `throughput.sh`.
This is not hypothetical. A change that cost 5–6% on `not_modified_304` and
`sidecar_hit_200` passed the full test suite, the lint gate and the mutation suite,
because **none of those measure speed**. Only running the benchmark caught it. If a change
touches the request path at all, run both harnesses.