# PLAN-cache.md — An eager content cache
## Where We Are
Serving one file, in `mini-static` 0.34.2, does all of this per request:
1. `mini-serve` splits and percent-decodes the path once and hands the segments over.
2. `check_segment` refuses any segment containing `/`, `\` or NUL, and any segment equal to
`..`.
3. `has_hidden_segment` refuses a dot-prefixed segment, exempting `.well-known` at the root,
unless `with_hidden_files()` was called.
4. `join_segments` pushes each segment onto the canonicalised root.
5. `File::open`, then `real_path_of(&fd)` and `starts_with(root_canon)` — **containment is
proven on the opened descriptor**, which is what removed the check-to-open window in
0.31. A directory retries with `index.html` appended, verified on its own descriptor.
6. `metadata()` from that descriptor.
7. Unless HTML injection applies, a `Range` was requested, or `without_precompressed()` was
called: up to two more `File::open` attempts, for `<path>.br` and `<path>.gz`.
8. An ETag from length and mtime; `If-None-Match` answers `304`.
9. The body — read inline at or below `INLINE_BODY_BYTES`, streamed in bounded chunks above
it — plus HTML injection for SPA or live-reload at or below 8 MiB.
10. Seven headers: `x-content-type-options`, `content-type`, `content-length`,
`cache-control`, `vary`, `etag`, `accept-ranges`.
**Every request opens and stats a file the process has already served thousands of times.**
Measured on this machine (Mac16,2, one worker each, `oha` c=50, interleaved, three reps, on
`styles.css`, 14 KB, from a real site):
| disk path, sidecar probe on | 55,658 | 17.96 |
| disk path, `without_precompressed()` | 65,134 | 15.35 |
| nginx, no `gzip_static` | 66,463 | 15.05 |
| **a realistic cached hit** (`examples/bench_cached.rs`) | **169,554** | **5.90** |
| a fixed response, no lookup at all | 196,360 | 5.09 |
So roughly **9.5 µs of each request is filesystem work**, and a lookup, all the segment
refusals, an ETag comparison and all seven headers together cost **0.81 µs**.
Three further facts the design has to live with:
- **15 guarantees are mutation-verified**, and `verify-guarantees.sh` exits non-zero if any
mutation is not caught by its test.
- **Live-reload watches the served root** and pushes SSE on change. It exists because files
under the root do change while the server runs.
- **The development filesystem is case-insensitive.** `/styles.css`, `/STYLES.CSS` and
`/Styles.Css` all serve the same 14,529 bytes today, because folding is delegated to the
operating system. Linux would answer `404` for the latter two.
## Where We Need To Be
An **opt-in** eager cache, with these observable behaviours:
- Calling the builder reads every eligible file under the root at construction into memory,
bounded by a byte cap, and answers subsequent requests for those paths from memory.
- **A cached response is byte-identical to the uncached one** — same status, same seven
headers (excluding `Date`), same ETag, same `304` on a matching `If-None-Match`, same
bytes for a `Range` request.
- **Every refusal in steps 2, 3 and 5 above still applies on a cache hit**, each proven by
its own mutation: `..`, encoded separator, backslash, NUL, dot-prefixed segment, and
containment.
- **Population refuses** anything that is not a regular file, anything whose opened
descriptor resolves outside the root, and anything beyond the byte cap.
- **A fifo under the root does not hang construction.**
- **Combining the cache with live-reload fails at construction**, in either builder order,
rather than serving stale content.
- **A path not in the cache falls through** to the existing disk path and behaves exactly as
it does today.
- Measured in the same interleaved harness: **cached hits at or above 150,000 req/s** on the
reference file, against 65,134 for the disk path.
## Commits
### 1. Refuse what must not be cached
- Does: adds a single eligibility check — given an opened descriptor, its metadata, the
canonical root and the remaining byte budget, decide whether this file may be cached.
Refuses a non-regular file, a descriptor whose real path is outside the root, and anything
that would exceed the budget.
- Verifies: unit tests over a fixture root — a fifo is refused, a symlink pointing outside
the root is refused, a symlink *inside* the root is accepted, an oversized file is refused,
an ordinary in-root file is accepted. Nothing calls it yet, so the whole existing suite
must pass unchanged.
- Touches: `src/cache.rs` (new, 96 LOC), `src/lib.rs`, `tests/unit/cache.rs` (168 LOC),
`Cargo.toml` (`libc` as a unix dev-dependency — the FIFO fixture needs `mkfifo`, which std
does not expose, and the crate itself calls libc only on macOS).
- Reverts cleanly: yes — no caller.
- **Grew to include the walk, and had to.** The predicate alone cannot prove the property that
matters: that the walk does not *descend* a symlinked directory. `is_cacheable` refusing the
link is necessary and not sufficient. Enumeration is therefore one thing — "which files may
be cached" — and both halves land together.
- **Carries `#![allow(dead_code)]`, removed in commit 2.** Nothing in the crate calls the
module yet, which is the point of landing it first, but an unreachable private module fails
`-D warnings`. `expect(dead_code)` would be the better marker, because it fails the build
once a caller exists and so cannot outlive its purpose — it does not work here, because
dead-code liveness differs between `cargo build` and `cargo clippy --all-targets` (the unit
tests are a caller in one configuration and absent in the other), so no single expectation
holds in both.
- **Why first.** This is the security core of the whole plan, and it is the half that a
benchmark cannot check. Landing it alone means it can be got right, and mutated, before a
single byte is cached.
### 2. Populate at construction, serve nothing from it
- Does: a builder that walks the root once, applies commit 1's check to each entry, reads the
eligible ones into a read-only map held inside the existing `Arc<Server>`, and stops at the
byte cap. **The serving path does not read the map.**
- Verifies: a test asserts the populated key set and total byte count exactly, for a fixture
containing a dotfile, a fifo, an out-of-root symlink, a nested directory and an oversized
file — and the test has a deadline, so a fifo hanging construction fails rather than hangs.
A second test asserts the byte cap truncates population rather than exceeding it. The whole
existing suite passing is what proves serving is unchanged.
- Touches: `src/cache.rs` (+160), `src/server.rs` (field and builder, +47), `tests/unit/cache.rs`
(+187), `tests/server.rs` (+55).
- Reverts cleanly: yes — the map has no readers.
- **Stores `fs::Metadata`, not a derived ETag.** The serving path builds its ETag with
`generate_etag(&metadata)`, so a cached response calls *the same function on the same input*
and cannot drift from an uncached one. Deriving the ETag here would have been a second
implementation of a value that must match exactly.
- **Sibling presence costs a set lookup, not two `open()` calls.** The walk has already been
past every path, so whether `<path>.br` exists is a membership test against the enumerated
set — which is why it is recorded at population rather than at serve time.
- **The non-UTF-8 key test is Linux-only, and not by preference.** APFS and HFS+ reject a
filename that is not valid UTF-8 (`EILSEQ`), so the fixture cannot be created on the machine
this crate is developed on. On macOS the property is type-level instead — the key is a
`PathBuf`, and no lossy conversion appears in `populate` — which is reviewable rather than
testable, and is marked as such in the test.
- **Three items carry `#[allow(dead_code)]` until commit 5**: `CachedFile::bytes`,
`CachedFile::metadata` and `ContentCache::get`. `has_precompressed_sibling` does not — the
construction log reports how many cached files have one, which is genuinely useful, since a
root reporting zero is one where `without_precompressed()` costs nothing.
- **The walk does not follow directory symlinks.** Paths beneath one fall through to the disk
path, which verifies them per request. Following them would mean re-deriving containment
for a whole subtree during a walk, which is the second-resolution-path hazard at its worst.
### 3. Make the live-reload combination unrepresentable
- Does: a server configured with both the cache and live-reload **refuses to start**.
- **The plan first said "fails at construction", which this API cannot express.**
`with_live_reload()` returns `Self`, not `Result` — so a builder called before the cache
cannot report a conflict, and one called after cannot be reached. Making either fallible is a
breaking change to a published signature for one diagnostic. The check goes instead at the
fallible boundary that already exists — `run_on`, and the `into_app` path beneath it — so the
refusal lands before the listener accepts anything, in either builder order, with no
signature change.
- Verifies: a test asserts `run_on` returns `Err` for both builder orders and succeeds with
either feature alone; a mutation removing the check fails that test.
- Touches: `src/server.rs` (+60), `tests/server.rs` (+90).
- Reverts cleanly: yes.
- **The plan missed an entry point, and it is the one that matters most.** `into_fallback` is
public, so a composed static-plus-API deployment never calls `run_on` and would have bypassed
the check entirely — in exactly the configuration the layering work exists to support. There
are three ways to start serving, so the rule is consulted from three places:
1. `with_content_cache` became **fallible**, catching the conflict at the call that created it.
Free to do: the method is new and unpublished, so no signature was broken.
2. `run_on` refuses to start, catching the reverse order, since `with_live_reload` returns
`Self` and cannot report anything.
3. `into_fallback` cannot return an error at all, so the handler it yields **fails every
request with a `500` naming the misconfiguration**. A bug found in the first minute of
testing beats stale content found in production weeks later.
- **One condition, three callers.** `Server::cache_conflict` is the single implementation; three
copies of a rule that must agree is the shape this codebase has been bitten by twice.
- Also removed a relic found in the area: `into_app`'s doc comment had been stranded above
`into_fallback` by the layering commit, so the two ran together into one confusing block.
- **Must precede commit 5.** Serving from a cache while a watcher reports the files changing is
exactly the stale-content bug this plan exists to avoid, and a comment saying "do not do
that" is not a defence. Same move as credentialed-wildcard CORS being rejected rather than
documented.
### 4. Make the response path work without a descriptor
- Does: a pure refactor. The response path already has an in-memory branch — `transformed:
Option<Bytes>`, used for HTML injection — and ranges, HEAD and the ordinary body all handle
it. What it does *not* have is a path where there is no `std_file` at all: the seek is
guarded by `transformed.is_none()`, and the streamed body takes the descriptor
unconditionally. This commit makes the body source explicit — bytes in memory, or a
descriptor — so nothing downstream assumes a file is open.
- Verifies: **the entire existing suite passes unchanged, and the criterion benchmarks show
no regression.** A pure refactor's verification is that nothing observable moved; the
benchmarks are part of that, since threading an enum through a hot path is exactly where
a regression would hide — measured against a baseline saved *immediately* before the change,
since this machine drifts thermally by over 10% across a session and a stale baseline would
report the drift as the refactor's cost.
- **The sidecar branch holds a descriptor too.** `select_precompressed_sidecar` returns an
open `File`, so "the body may be bytes rather than a descriptor" has to cover that branch as
well, not only the main one.
- Touches: `src/server.rs` (~85 LOC changed, net −4), `benches/` baselines.
- Reverts cleanly: yes.
- **Result: no regression, and one small improvement.** Against a baseline saved minutes before
the change: `small_file_200` −0.56% (p = 0.02), `large_file_200` −1.3% (p = 0.32),
`not_modified_304` −0.42% (p = 0.15), `sidecar_hit_200` −0.27% (p = 0.33). The only
significant movement is *faster*, which is consistent with what the refactor removed: the
seek was guarded by `transformed.is_none()` on every ranged request, and now lives in the
descriptor arm where it cannot be reached without a descriptor.
- **The invariant became the type.** The old pair — `Option<Bytes>` plus a `File` — carried
"exactly one of these is the real source" by convention, enforced by that guard. `BodySource`
has no state where both or neither is present, and the descriptor is moved into exactly one
branch of a single `if`, so there is no impossible state left to assert against (A4 by
construction rather than by tripwire).
- **Why this is its own commit.** Discovered while resolving an open question: the body
construction is already shared and ready for cached bytes, but the descriptor is assumed
present across ~200 lines. Folding that restructuring into the commit that starts serving
from the cache would mix a refactor with a behaviour change, and a differential test would
then be checking two things at once.
### 5. Serve cached entries that have no sidecar siblings
- Does: on a hit, answer from memory — the same seven headers, the same ETag, the same `304`,
the same bytes for a `Range`. A file with a `.br` or `.gz` sibling is **not** cached by this
commit and falls through to the disk path unchanged.
- Verifies: a **differential test** — for every entry in a fixture root, the cached and
uncached responses are compared field by field and must be identical. The fixture must hold
the awkward cases or "identical" is proven only for the easy one: **a directory served via
its index, a `304` from `If-None-Match`, a satisfiable `Range`, an unsatisfiable `Range`, a
`HEAD`, a hidden file, a non-UTF-8 filename, and a path present on disk but absent from the
cache.** Then each existing refusal is re-proven on the hit path with its own mutation id:
`..`, encoded separator, backslash, NUL, hidden segment, containment.
- **Diff the whole header set and fail on any unexpected difference.** "Every header except
`Date`" is an assumption; a hardcoded skip list would hide a header the cache forgot.
- **Also verifies the number, because no other commit does:** in `bench/faceoff.sh`,
interleaved, cached hits at or above **150,000 req/s** against the disk path's ~65,000. If
the hit path does not clear that, the plan has not delivered its reason to exist and stops
here rather than proceeding to commit 6.
- Touches: `src/server.rs` (+~75), `src/resolve.rs` (+~25), `tests/cached_serving.rs` (new, 290),
`examples/bench_server.rs` (+7).
- **Result: 162,035 req/s mean against the 150,000 gate** (163,315 / 164,421 / 158,369,
interleaved with the disk path's 63,281 mean). About **2.4x nginx**, and 4% under the
prototype's 169,554 — the difference being the refusals and key-building the prototype
skipped.
- **The differential test found a hidden-file leak within a minute of the cache first
answering.** `/.env` was refused from disk and **served from memory**: the lookup bypassed the
hidden-file check entirely. Fixed by extracting `resolve::servable_segments` and calling it
from both paths — one implementation cannot diverge from itself. This is precisely why the
plan made a differential test the verification rather than "check the cache works".
- **Mutation found the production path was the untested one.** The differential test initially
ran only through `handle_request`; removing the refusal from the `respond` branch — the path
`mini-serve` actually uses — went undetected. Every case now runs through both entry points.
- **Mutation also found dead policy and deleted it.** An earlier draft declined to cache any
file with a `.br`/`.gz` sibling. Removing that decline left every test green, because the
sidecar probe runs afterwards and replaces the body anyway — the code was inert. Dropping it
is *faster* too: a client sending no `Accept-Encoding` now gets such a file from memory.
- **Two mutations were uncatchable until the tests changed.** The directory-index retry and the
cache being consulted at all are invisible while the files are still on disk, since a miss
falls through and serves the same bytes. Both are now proven by removing the file from disk
after construction, so only a server answering from memory can succeed.
- Reverts cleanly: yes — the map returns to having no readers.
- Excluding files with siblings keeps this commit honest: a site that ships precompressed
assets gets today's behaviour rather than a silent loss of it.
### 6. Cache the precompressed variants
- Does: population records `.br` and `.gz` siblings alongside the plain body; the hit path
chooses between them with the existing `Accept-Encoding` quality logic. A cached root then
needs no probe at all.
- Verifies: a cached root containing sidecars serves them with the same `Content-Encoding`
and ETag as the uncached path, and `accept-encoding: identity` gets the plain body. Plus
the decisive one: **after construction, move the whole root aside, then serve** — the test
fails if the hit path touches the filesystem at all. Renaming rather than `chmod`, because
the owning user can traverse a mode-`000` directory on some systems and the test would then
prove nothing.
- Touches: `src/server.rs` (+~70), `tests/cached_serving.rs` (+130).
- Reverts cleanly: yes.
- **No extra storage was needed.** A `.br` file is a regular file, so population had already
enumerated it under its own name — serving a variant is a second lookup in the same map, not
a second copy of anything. A cached root spends **zero `open()` calls** on content negotiation
where the disk path spends up to two.
- **`preferred_encodings` is shared** between the disk probe and the cache lookup, so the two
cannot disagree about which encoding a client asked for. Content negotiation is no safer a
place for two implementations of one rule than path decoding was.
- **A mutation found a real divergence in this commit's first draft.** The disk probe was guarded
by `cached_key.is_none()`, so a cached hit never looked on disk for a variant. But budget
truncation can hold `app.css` without holding `app.css.br` — so a cached server would have
served an unencoded body where an uncached one serves a compressed one. Guard removed, with a
test using a 150-byte budget that holds the plain file and truncates before the variant.
- **A mutation also exposed a gap that predates this work.** Deleting the quality sort from
content negotiation passes the *entire* suite: nothing in the crate verified that
`br;q=0.5, gzip` serves gzip rather than brotli. The differential test could not catch it
either — a bug both paths share shows up as agreement. There is now an absolute assertion, and
it is the second time in this commit that differential testing proved insufficient on its own.
### 7. Record the guarantees and the number
- Does: threat-model rows for the hit-path refusals and the population refusals; a cached row
in `bench/faceoff.sh`; the trade stated in `README.md`.
- **The "Not defended" section gains the promise itself:** with the cache on, this crate
serves what it read at startup. A file replaced in place, or torn by a concurrent write
during the walk, is served in that state until the process restarts. That is a guarantee
*given up* for throughput, and it belongs on the list of things this crate does not defend
rather than in a doc comment.
- Verifies: `verify-guarantees.sh` catches every mutation including the new ones; the
citation test and the mutation-id drift test pass; the harness prints the cached row.
- Touches: `THREAT_MODEL.md`, `verify-guarantees.sh`, `bench/faceoff.sh`, `README.md`.
- Reverts cleanly: yes.
## Where To Stop
- **After commit 2** the cache exists and nothing serves from it. No behaviour change, no
risk, and the dangerous half is verified. This is a safe place to abandon.
- **After commit 5** the win is real for every root without precompressed siblings — which is
every root in this ecosystem today, since `mini-build` does not produce them.
- Commits 6 and 7 are completion, not correctness.
## Open Questions
- ~~**The builder's name.**~~ **Decided: `with_content_cache(max_bytes)`.** Mechanism-named,
with the promise in the doc comment rather than the signature.
- ~~**The byte cap: value, and whether it is a parameter.**~~ **Decided: a required parameter,
and exceeding it truncates with a log line** rather than refusing to start. Refusing would
turn a growing site into an outage for a performance feature, and with the operator naming
the number, spending it and reporting what was spent is the honest behaviour (A11). Sorted
enumeration makes that truncation deterministic.
- **Case-insensitive filesystems.** `/STYLES.CSS` serves today and would miss an exact-key
cache, falling through to disk — correct, but the hit rate becomes filesystem-dependent,
and no test on macOS can observe the Linux behaviour. Normalising keys is worse: it risks
serving a file for a path that must `404` on a case-sensitive system. Recommendation: exact
keys, documented, and a note in the threat model that this is a performance property rather
than a correctness one. **Confirm before commit 5.**
- ~~**`Range` requests on a cached body.**~~ **Resolved before grilling.** They share the
construction: `transformed: Option<Bytes>` already carries an in-memory body, and the range
branch slices it (`bytes.slice(start..=end)`) while HEAD empties it. So cached bytes need no
new body logic. What the path *does* assume is that `std_file` exists, which is why commit
4 now exists to remove that assumption.
- **The 404 page** is read from disk on every miss and is a natural cache entry. Out of scope
here; worth its own commit afterwards.
- **`with_immutable_assets`** sets `Cache-Control` and is orthogonal to this cache. Named only
so the similar wording does not suggest a connection.
## Grill Verdict
Round: 1
Status: PASS
Findings resolved:
- **[2, G1] The cache key was never specified** — the one decision the whole plan turns on.
Now fixed in commit 2 as byte-exact path segments rather than a `String`, with three reasons
each of which would be a defect: a lossy conversion maps different files onto one key, so a
request could be served another file's bytes; the key must be the exact inverse of what
`mini-serve` hands over; and the `index.html` retry has to be in the hit path or `/` — the
most common request on any site — misses the cache entirely.
- **[3, G1] The plan asserted an error the API cannot express.** `with_live_reload()` returns
`Self`, so no builder can report the conflict. Moved to `run_on`, the fallible boundary that
already exists, rather than making a published signature fallible for one diagnostic.
- **[1, G2] The eligibility predicate bundled three unrelated refusals.** A byte budget is a
policy about totals, not a property of a file; keeping it there made the predicate
undescribable in one sentence and made "oversized is refused" ambiguous — refused for its own
size, or because the budget ran out? Accounting moved to commit 2.
- **[5, G1] Commit 5 needed data commit 2 did not collect** — whether a file has a `.br`/`.gz`
sibling. Population records it, since the walk already reads the directory and commit 5 could
not obtain it without walking again.
- **[5, G1] The differential test's fixture was unspecified**, so "byte-identical" would have
been proven only for the easy case. Now enumerated: directory index, `304`, satisfiable and
unsatisfiable `Range`, `HEAD`, hidden file, non-UTF-8 filename, and an uncached path.
- **[5, G1] No commit verified the throughput target.** It sat in "Where We Need To Be" with
nothing attached. Commit 5 now owns it at ≥150,000 req/s, and stops the plan if unmet.
- **[5, G3] "Every header except `Date`" was an assumption.** The test diffs the full set and
fails on any unexpected difference, so a header the cache forgets cannot hide in a skip list.
- **[1, G1] A torn read becomes permanent** under eager caching, where today it costs one
request. Stated in commit 2 and added to the threat model's "Not defended" list in commit 7.
- **[4, G1] The sidecar branch also holds a descriptor**, so the refactor must cover it too.
- **[4, G3] "No regression" needed a drift-proof baseline** — this machine moves >10% across a
session, and a stale criterion baseline would report the drift as the refactor's cost.
- **[6, G3] "Make the root unreadable" would have proven nothing** — the owning user can
traverse a mode-`000` directory. The test renames the root aside instead.
- **[2, G1] A mid-walk I/O error had no stated outcome.** It skips the entry and the path falls
through to disk; failing construction would make an unrelated permissions problem fatal.
Findings accepted as tradeoffs:
- **Case-insensitive filesystems reduce the hit rate, and no macOS test can observe it.**
`/STYLES.CSS` serves today and will miss an exact-key cache, falling through to disk —
correct, but the hit rate becomes filesystem-dependent. Normalising keys is worse: it risks
serving a file for a path that must `404` on a case-sensitive system. Accepted as a
performance property rather than a correctness one, and recorded in the threat model.
- ~~**`read_dir` order becomes observable once the budget can be exhausted.**~~ **No longer a
tradeoff.** Enumeration is sorted, so the byte budget truncates a deterministic list: two
servers on identical roots cache the same subset regardless of the order the files were
created. Cost was one comparison sort at startup. The 65,536-entry *safety* ceiling is the
one exception — it returns the first entries encountered, then sorted, because collecting
every path before truncating is the unbounded work that ceiling exists to prevent. That is
documented at the return site.
- **Seven commits for a performance feature**, in a crate whose pitch is auditability. The count
is driven by the security half — the predicate, population, and the refusal to start each
land separately because each is independently verifiable and independently revertible.
Recorded so it is entered deliberately.