Expand description
The on-disk cache: one file per file, keyed by a hash of everything that determines a summary.
The rules this module implements are the ones fixes; what follows is the part a reader of the code needs to know to change it safely.
§Why one file per summary
A single blob for the whole project would have to be rewritten whenever any file changes — for a large project that is hundreds of megabytes of I/O per keystroke, and a crash while writing loses everything. One file per summary makes invalidation local, writes independent, and a torn write cost exactly one file.
§What the key is made of, and why each part is in it
content_hash what the file says — a change here changes the summary
context_hash compiler settings and the file's own directory; see below
format_version the producer — a parser or schema change makes every summary wrong
reading_fingerprint the readers themselves — measured, not remembered; see [`READING_FINGERPRINT`]context_hash is the configuration and the directory the file sits in, which are one question rather than
two: #include "widget.h" means the one beside me, so the same text in two directories describes two
different compilations and resolves to two different files. The directory was missing from the key at first, and
the hole was invisible until a test put the same text in two directories: the two files shared one entry, and
the second one was handed the first one’s resolved includes — a jump to a header it does not include.
The part of the key that is not in it is which file it is: nothing here names a path, only the directory the
path is in. That is what makes a branch switch cheap — see content_hash.
format_version is deliberately the blunt instrument: the grammar is still moving, and a summary written by a
different parser is not “probably still fine”, it is a silent wrong answer. Bumping the number is the one
invalidation that must never be forgotten, so it lives in the key rather than in a migration step.
It was forgotten, for many rounds — which is why the key also carries a fingerprint of the readers
(READING_FINGERPRINT), taken from the versions the build already carries: a rule that depends on remembering is
a rule that gets missed, and this one is a constant of the package.
§Why the macro environment is not in the key
A fourth part used to be here: the set of content hashes of the files that define macros on the way in. It was
the one part that could not be computed from the text and the path — the file’s own #defines and its own
#includes are products of the parse — and that had a consequence the design did not intend: a lookup could
not happen until after the parse, so the disk cache never saved the thing it exists to save. It saved writes.
It is also not needed, and the evidence is a chain of three facts about the producer:
crate::build_scopestakes only the syntax tree, so declarations are collected from the whole file;crate::build_factstakes the preprocessing state to find where the directives are, not to decide which branch was taken — every fact records the#ifit was written in as acrate::FactGuardand lets the consumer decide, which is the whole reason guards are stored rather than applied;#include MACRO— the one directive whose meaning depends on expansion — is never resolved (crate::include’s resolver returnsUnresolvedfor it, and says so), so a file that writes one is not stored at all.
So a summary is a function of the text, the directory, the configuration and which candidate headers exist —
not of which macros were in force. If that ever stops being true — macro expansion inside declarations, or
pruning the untaken branch of an #if — the environment has to come back here and FORMAT_VERSION has to
be bumped in the same change, because summaries already on disk would be silently stale rather than merely
unreachable.
§Why the hash is written here rather than taken from std
DefaultHasher is explicitly allowed to change between Rust releases, and a cache key that changes when the
compiler is upgraded is a cache that silently misses forever (or worse, hits with a different meaning if the
value is ever persisted). FNV-1a is fixed by definition, is a few lines, and is strong enough for this job: the
key guards a local cache, and a collision costs a stale summary for one file — a wrong answer a re-parse
fixes, not a security boundary.
Modules§
- cache_
db - The directory this replaces held 1 854 files / 160.6 MB on a real project: 1 768 summaries (69 MB), 85
.tufiles (85.6 MB) and a 6 MBsharing.bin. Three costs came with that shape, and each is what a database removes: - cache_
lru - A cache that forgets: the entries a session keeps, and the point past which the oldest goes.
Structs§
- Summary
Key - Everything that determines what a summary of one file contains.
Constants§
- CACHE_
BUDGET_ BYTES - How much the summary cache may hold before a sweep starts removing the oldest entries.
- CACHE_
DIRECTORY - The directory name this crate keeps its cache in, under the project root.
- FORMAT_
VERSION - The version of everything that produces a summary: the parser’s grammar, this crate’s summary schema, and the meaning of the fields in it.
- READING_
FINGERPRINT - The fingerprint of the readers a summary was built by, in the key.
Functions§
- content_
hash - The hash of a file’s text, as the cache sees it.
- fnv1a64
- A stable 64-bit FNV-1a hash of
bytes.