# PLAN-perf.md — Take filesystem work off the blocking pool
## Where We Are
Under load (`oha`, c=50, one worker each, identical files), `mini-static` 0.29.1 serves
a small file at ~17,800 req/s using 380% CPU — 4,500 requests per CPU-second — against
nginx's ~52,700 req/s at 90% CPU, 58,300 per CPU-second. nginx is 2.8x the throughput at
11x the efficiency.
The cause is measured, not guessed: four spikes, each isolating one change, re-measured
under the same load (three runs each):
| baseline 0.29.1 | 17,800 | 380% | 4,500 |
| + resolve inline (no `spawn_blocking`) | 24,000 | 295% | 8,100 |
| + `open`/`stat` sync inline | 30,500 | 185% | 16,500 |
| + small bodies read sync inline | 30,200 | 148% | 20,400 |
| + sidecar probe sync inline | 36,000 | 98% | 36,700 |
Every line is the same defect: filesystem work dispatched to tokio's blocking thread
pool, where the dispatch costs more than the work. Four separate sites:
1. **Path resolution** goes through `spawn_blocking` per request (`src/server.rs:903`),
plus a `Server::clone()` and a `String` allocation to move into the closure. This
echoes the `mini-build` finding that Rust's process-spawn overhead dwarfed the tool
it launched — same shape, thread-pool edition.
2. **`File::open` + `metadata`** use `tokio::fs`, which is itself `spawn_blocking`
internally — the pool round-trip survived commit 1's fix in disguise.
3. **Small response bodies** stream through `FileBody` over a `tokio::fs::File`: one
pool round-trip per 64 KiB chunk, so a 575-byte file pays the full dispatch for one
chunk of real work.
4. **The sidecar probe** makes up to two `tokio::fs::File::open` attempts per request
for `.br`/`.gz` files that usually do not exist — and browsers send
`Accept-Encoding` on every request, so this is the common path, not a corner.
A `WORKERS=1` server consuming 3.8 cores is these four sites: one core of real work,
three of pool dispatch and handoff.
**This plan reverses a documented design decision.** The README's performance section
presents the `spawn_blocking` resolution as a feature: "a slow filesystem lookup for one
request no longer stalls every other task." That reasoning was sound for its premise —
a filesystem where `canonicalize` can block for milliseconds (NFS, a spun-down disk) —
and the premise does not hold for the deployment this crate targets, a local SSD where
these calls are single-digit microseconds. The premise change must be documented as
loudly as the original decision was.
## Where We Need To Be
- Small-file serving at or above 35,000 req/s at ~100% of one core in the same harness —
the spike numbers reproduced by the committed implementation, not approximated.
- Filesystem metadata operations (`canonicalize`, `open`, `stat`, sidecar probe) run
inline on the request task. Bodies at or below a named threshold are read inline and
served buffered; larger bodies still stream through `FileBody` in bounded chunks, so
memory per in-flight response stays bounded exactly as before.
- The README's performance section describes the new policy and its stated tradeoff:
serving from a filesystem with unbounded latency (network mounts) now stalls
concurrent requests on the same worker, and this crate optimizes for local disk.
- `benches/handle_request.rs` reflects the change, with before/after recorded.
- No behavioral change: every existing test passes unmodified. Identical bytes,
identical headers, identical status codes — this plan touches only *where* the work
runs.
## Commits
### 1. Resolve inline; delete the `spawn_blocking` dispatch
- Does: Replace the `spawn_blocking(move || server.resolve(...))` block with a direct
`self.resolve(request_path)` call, deleting the per-request `Server::clone()` and
`String` allocation with it. Update the function's doc comment and the README
performance bullet that documented the old design, stating the premise change rather
than silently dropping the claim.
- Verifies: full suite passes unmodified; the load harness shows ~+35% req/s and ~-85
points of CPU against a baseline measured the same day on the same machine.
- Touches: `src/server.rs` (~15 LOC removed), README.
- Reverts cleanly: yes.
### 2. Open, stat, and sidecar probe synchronously
- Does: `std::fs::File::open` + `metadata()` inline, then `tokio::fs::File::from_std`
for the streaming path; same for the sidecar probe's two candidate opens. The probe
keeps returning an open handle (not a stat-then-open), preserving the
no-TOCTOU-between-probe-and-serve property it has today.
- Verifies: suite passes unmodified — sidecar tests included; harness shows the
combined ~16,500 → ~36,700 req/CPU-s step alongside commit 3.
- Touches: `src/server.rs` (~20 LOC).
- Reverts cleanly: yes.
### 3. Buffer small bodies read inline
- Does: Bodies with `metadata.len() <= INLINE_BODY_BYTES` (64 KiB — one `FileBody`
chunk, so the threshold is "would have been a single chunk anyway") are read with
`std::fs::read` and served `Buffered`; larger bodies keep the streaming path
untouched. Named constant with the reasoning on it.
- Verifies: suite passes unmodified — `content_length` and large-file streaming tests
cover both sides of the threshold; harness confirms the CPU step.
- Touches: `src/server.rs` (~15 LOC).
- Reverts cleanly: yes.
### 4. Re-measure and record — **DONE**
- Does: Re-run the load harness and `benches/handle_request.rs`; update the bench
module docs' baseline table and the README perf section with measured before/after.
- Verifies: the committed implementation reproduces the spike numbers within noise; if
it does not, that gap is investigated before this commit lands, not after.
- **Result: the committed version beat the spike** — 45,100 req/s @ 97% CPU (46,200
req/CPU-s) against the spike's 36,000 @ 98%. The difference is commit 3's final form:
the spike re-opened the file by path (`std::fs::read(&path)`), where the committed code
reads from the already-open handle — one fewer open per request, and no reopen window.
- **The suite caught a real bug the spike missed.** The spike's buffered branch read from
`path` — the *original* file — while a selected sidecar's handle and metadata pointed
at the sidecar. Wrong bytes, invisible in the spike fixture (no sidecars on disk),
caught immediately by `precompressed_sidecar_served_when_accept_encoding_matches`.
The fix (read from the handle) is also the better design. Sidecar selection now
traffics in `std::fs::File` and converts to async only at the streaming branch.
- Final table (same day, same machine, `oha` c=50, one worker each):
| | req/s | CPU | req/CPU-s | p50 |
|---|---|---|---|---|
| mini-static 0.28.0 | 17,800 | 380% | 4,500 | 2.7 ms |
| mini-static 0.30.0 | **45,100** | **97%** | **46,200** | 1.09 ms |
| nginx (1 worker) | 54,600 | 91% | 59,900 | 0.84 ms |
2.5x throughput, 10x per-CPU efficiency, from 8% of nginx's efficiency to ~77%.
- Touches: `benches/handle_request.rs` docs, README.
- Reverts cleanly: yes — docs only.
## Open Questions
- **The blocking-inline tradeoff is accepted, not eliminated.** On a pathological
filesystem (NFS stall, dying disk) an inline `canonicalize` blocks every connection
sharing that worker, where the old design isolated it to the pool. Accepted because:
the target deployment is local SSD behind a CDN; the numbers say the pool costs 3x
the work it shelters; and an embedder who needs isolation can run a multi-threaded
runtime. Documented in README rather than made configurable — a knob would be two
code paths, double the test surface, for a premise this crate does not serve.
- **What this plan does not chase:** the remaining ~40% gap to nginx per CPU-second is
syscall count — `canonicalize` walks the path on every request where nginx has an
open-file cache, and nginx writes bodies with `sendfile` where hyper cannot. Both
are real; both are caching or zero-copy work with correctness surface (invalidation,
TOCTOU) that belongs to a future plan argued from these numbers, not bolted onto
this one.
## Grill Verdict
Round: 1
Status: PASS
Findings resolved:
- [C1, G3] The old design's rationale lives in the README as a *feature*; silently
inverting it would leave the docs claiming the opposite of the code. Both C1 and the
plan preamble now require the README change to state the premise shift explicitly.
- [C2, G1] A stat-then-open sidecar probe would introduce a TOCTOU window the current
open-then-serve shape does not have. C2 now names the property and preserves it
(sync open, `from_std` handoff).
- [C3, G3] The 64 KiB threshold looked arbitrary until tied to `FILE_CHUNK_SIZE`: at or
below one chunk, the streaming path did one read anyway, so buffering changes dispatch
only, never memory shape. Stated on the constant.
- [C4, G1] Spike numbers came from a hacked tree; nothing guaranteed the clean
implementation reproduces them. C4 exists to force the re-measurement before the plan
closes.
- [Plan, G2] A configurability knob for inline-vs-pooled was considered and rejected as
speculative generality; recorded in Open Questions with the reason.
Findings accepted as tradeoffs:
- [C1–C3, G1] Unbounded-latency filesystems now stall co-scheduled requests. Reason:
measured 3–8x efficiency cost paid by every deployment to shelter a deployment shape
(network-mounted roots) this crate does not target; escape hatch (multi-thread
runtime) exists; documented.