Expand description
Where the objects live between fetches, and who may read them.
A git store needs somewhere to put an object database. That directory holds
the contents of a private repository, so it is created private at
creation — 0700, by the call that creates it, never by a chmod
afterwards. A chmod after the fact leaves a window in which the directory
is world-readable, and a window is all an attacker on a shared host needs.
It is the same rule the core crate’s on-disk cache follows.
§Two places it can be
A temporary directory, by default. Made when the first fetch runs, removed when the source is dropped. Nothing survives the process, which is the right default for a container: no growth to manage, no stale objects, no secret left on a volume after the pod dies. It costs one full fetch per process start.
A directory the caller names, with GitSource::cache_dir. It survives
restarts, so a restart transfers almost nothing — worth it for a large
repository or a fleet that restarts often.
§What a store may delete, and where
Either directory grows: a shallow fetch of a moving branch writes one pack
per transfer, the old ones stop being reachable the moment the local ref
moves, and nothing in git removes them until a gc. A long-lived watcher
accumulates one pack per push for as long as it runs, and a caller-named
directory carries that on across restarts.
So this crate compacts, and the rule it compacts under is narrow enough to state in one line: it deletes only what it wrote, only in a directory it created for itself, and only on a trigger the caller can see and turn off.
- what it wrote — the working directory is a bare object database holding packs from this crate’s fetches and one ref this crate writes. There is nothing else in it to lose, and what is removed is re-obtained by the very fetch that removed it.
- a directory it created — a marker file is written when this crate
initialises the object database, and a directory without it is never
touched. A caller who points
cache_dirat a repository that already exists gets the old behaviour, unpruned, rather than losing a repository to a store that assumed the directory was its own. - a visible trigger —
Builder::compact_afteris the number of transfers a directory may accumulate, thirty-two by default, and0turns it off entirely for a caller who would rather rungit gc --prune=nowon their own cadence.
The compaction is not a gc: it empties the object database and lets the
next fetch refill it, which is one full transfer every compact_after
pushes and leaves the directory holding exactly the current commit again.
Deleting a pack and repacking would be less to transfer and much more to
get wrong — a store that rewrites an object database is a store that can
corrupt one, and this one cannot: the only thing it can lose is a copy of
something the remote still has.
It happens before a fetch that is going to read, never after one and never on a watch’s idle tick, so nothing the current call depends on is removed and nothing is emptied that is not about to be refilled.
§One directory, one source
Two sources fetching into one directory would interleave their ref updates and their packs. A caller-named directory is therefore claimed by the source that names it, and a second source in the same program naming the same directory is refused at construction — before anything is written — rather than corrupting it. The default, a temporary directory per source, cannot collide at all.
§Two processes are still not detected, and that is a decision
The claim above is in this process only. Two programs pointed at one directory are not detected, are not supported, and this crate will not grow a lock file to change that. The reasoning is worth writing down, because “add a lock file” is the obvious answer and it is the wrong one here:
- A stale-lock heuristic is wrong exactly where sharing happens. The deployment that shares a working directory is not two programs on one host; it is one volume mounted into two containers. Pid liveness cannot see across a pid namespace — both containers have a live pid 1, so a lock left by a dead process reads as held and a lock held by a live one reads as held for a different reason. An age bound replaces that with a guess about how long a fetch may take, and two nodes writing one network volume do not agree on the clock the guess is measured against.
- An advisory lock the kernel releases has no staleness — and no reach.
flockwould be the correct mechanism, and it is unreliable on precisely the network filesystems that make the sharing possible in the first place. It also costs a dependency in a crate whose small graph is one of the things it claims, to convert an unsupported configuration into a start-up failure on filesystems that do not implement locking.
What is done instead is to bound the damage. Concurrent fetches into one
object database are what git itself is built for: objects are written to a
temporary file and renamed, and a ref update takes a .lock. Compaction is
the part this crate added, and it is why the marker above is required: a
program that empties a directory another program is reading costs that
program one failed fetch, which is a failure this crate already promises
to survive — the previously fetched document stays installed and a watch
waits out the interval. That is asserted rather than claimed:
a_working_directory_emptied_by_another_program_costs_a_fetch_rather_than_the_source
empties one underneath a live source and fetches again. Give each program
its own directory anyway.