# PLAN-fd-containment.md — Check containment on the opened fd, not before it
## Where We Are
Every request proves containment by `canonicalize()` (a syscall per path component) and
then *separately* opens the file it just validated. Two consequences:
- **A TOCTOU window**, documented as accepted since the hardening plan: between
`canonicalize` and `File::open` the path can be swapped, so the file served is not
provably the file checked. Benign for a root nobody else writes to; still a gap.
- **Cost**: measured at 11.35 µs for the canonicalize + open + stat sequence on a
three-component path, against 7.04 µs for the fd-first equivalent — ~38% of the
metadata phase, roughly 15–20% of the whole request at current throughput
(45,100 req/s @ 97% CPU vs nginx's 54,600 @ 91% on the same day).
The fd-first equivalent: **open first, then ask the kernel what was opened** —
`fcntl(F_GETPATH)` on macOS/iOS, `readlink /proc/self/fd/N` on Linux — and run
`starts_with(root)` on that real path. The check lands on the very fd that will be
served, so there is nothing left to race: faster *and* strictly stronger.
Facts checked before this plan: `libc` is already in the dependency tree (tokio);
`resolve`/`resolve_with_canonical_root` are public API used by the property tests and
`benches/path_resolution.rs`; symlink semantics today are follow-then-verify
(`canonicalize` + `starts_with`), which `F_GETPATH` reproduces exactly.
## Where We Need To Be
- `handle_request` gets an opened `std::fs::File`, its `Metadata`, and the real path in
**one** resolver call, with containment verified on that fd. The separate
open-after-resolve — and its 500-on-open-failure path, which only existed because of
the window — disappears.
- Symlinks: followed, then the target must lie under the root — same policy, now
enforced on the fd. NUL, `..`-segment, and hidden-file prechecks are unchanged (they
act on the request path, before any filesystem contact).
- Directory handling unchanged: an opened directory (fstat says dir) retries with
`index.html` appended, verified the same way.
- Public `resolve`/`resolve_with_canonical_root` keep their signatures and pass the
existing 15 unit cases and 7 property tests **against the new mechanism** — they wrap
it and return the path, so the property suite exercises what production runs, not a
parallel legacy path.
- Non-macOS/Linux platforms fall back to the current canonicalize implementation behind
`cfg`, clearly marked as the portability path.
- Failure collapse preserved: any open failure is `NotFound` (no errno oracle);
containment failure is `Traversal`; both still render identical 404 bytes.
## Commits
### 1. fd-based open-and-verify in `resolve`, wired through `handle_request` — **DONE** (`b0fab74`)
- Does: Add `real_path_of(&File)` (cfg macos/ios: `F_GETPATH`; cfg linux:
`/proc/self/fd` readlink; else unavailable) and `open_within_root` returning
`{file, metadata, path}`. Reimplement `resolve_with_policy` on top (drop the fd,
return the path) so public API and property tests ride the real mechanism.
`handle_request` switches to the combined call; `libc` becomes a direct dependency
(one `fcntl` wrapper — already in the tree transitively).
- Verifies: all 15 resolve cases and 7 property tests pass unchanged — symlink
escape/allow included; full suite green; the load harness shows the step toward
~52–55k req/s.
- Touches: `src/resolve.rs` (~90 LOC), `src/server.rs` (~15 LOC), `Cargo.toml`.
- Reverts cleanly: yes.
### 2. Measure, record, update docs — **DONE**
- Does: Re-run the `oha` harness and criterion; update README (the security section's
TOCTOU note becomes "closed as of 0.31.0", the perf numbers refresh) and the bench
module docs.
- Verifies: committed numbers reproduce the probe within noise, or the gap is explained
before landing.
- **Result, same-run, one worker each, identical files (`oha` c=50):**
| | req/s | CPU | req/CPU-s | p50 |
|---|---|---|---|---|
| mini-static 0.30.1 | 45,100 | 97% | 46,200 | 1.09 ms |
| mini-static 0.31.0 | **56,700** | 98% | **57,900** | 0.87 ms |
| nginx (1 worker, sendfile) | 52,500 | 91% | 57,900 | 0.88 ms |
Ahead of nginx on throughput, level on per-CPU efficiency — without a cache. The
mutation check: disabling the `starts_with` on the fd's real path fails exactly the
two symlink-escape tests and nothing else, which is the correct blast radius, since
symlinks are the only route past the `..` segment pre-check.
- **The 500-on-open-failure path is gone as predicted**: a file vanishing after
resolution is now a 404 (it *is* a miss), and no test asserted otherwise — verified
by grep before the change.
- Touches: docs only.
- Reverts cleanly: yes.
## Open Questions
- **The fallback path is a real second implementation** (non-mac/Linux), which is the
duplication this repo usually refuses. Accepted: the alternative is dropping platform
support or losing the win; the fallback is the *old, tested* code, not new logic, and
the property tests run against whichever variant the platform compiles.
- **`openat2(RESOLVE_BENEATH)`** would make escape impossible rather than detected on
Linux ≥5.6 — strictly better, but a third variant and a runtime kernel-version probe.
Deferred until Linux is a measured deployment target; noted here so it isn't
re-derived.
## Grill Verdict
Round: 1
Status: PASS
Findings resolved:
- [C1, G3] Reimplementing the public functions *on top of* the fd mechanism (rather
than leaving them canonicalize-based) is load-bearing: otherwise the property suite
would validate a path production no longer takes. Stated in Where We Need To Be.
- [C1, G1] Open errors must stay collapsed to `NotFound` — an errno-differentiated
response would be a fresh existence oracle. Stated.
- [C1, G1] The 500-on-open-failure path exists only because of the resolve/open window;
its removal is a behavioral improvement (a vanished file is a 404, not a 500) and the
affected expectation lives in no test — verified by grep before writing this.
- [C1, G3] Directory retry must re-verify the joined `index.html` on its own fd, not
trust the directory's verification. Stated in Where We Need To Be.
Findings accepted as tradeoffs:
- [C1, G2] Platform-specific code with a fallback. Reason above, in Open Questions.