Expand description
Computing a git delta, for the one entry shape this engine cannot copy: a stored delta whose base falls outside the request and whose receiver holds nothing. The only place in this crate that computes one. Computing a git delta, for the one entry shape this engine cannot copy.
§Why an engine whose whole claim is “it copies” computes anything at all
crate::git_ops::GitStore::emit_set copies stored pack entries byte for
byte, and that is the product. One shape defeats it: a stored delta whose
base falls outside the request. The base cannot be added — a base pulled
into a narrowed clone drags an object the client never asked for, and if it
is a tree it arrives owing children the pack does not carry, which is the
did not receive expected object defect §2 of the standing report records.
Naming the base by oid in a REF_DELTA works for a fetch, where the
receiver already holds it, and cannot work for a clone, where the receiver
holds nothing.
That left exactly one answer for a clone: send the object whole. Measured
on h2h-linear-sha1-2048c-1024f-16k, that is 961 of 31 805 entries — 3.0 % —
and it doubled the wire: 11 613 666 bytes where stock git sends 5 820 000 for
the identical object set. Three per cent of the entries were essentially all
of a 2× gap, because the 961 are 16 KiB blobs whose stored delta is 284 bytes.
So this module exists to answer the third option: re-delta the boundary
entry against something the pack DOES carry. It is the only place in this
crate that computes a delta, crate::git_ops::GitStore::whole_entry is the
only other place that deflates on a serving path, and both run on the same
boundary fraction and nowhere else.
§The receipt stays a fact
An entry this produces is crate::pack_walk::EmitEntry::recompressed, same
as the whole rebuild it replaces — its payload really was inflated and
re-deflated, and copied + recompressed == objects is unchanged. What is new
is crate::pack_walk::EmitEntry::deltified, a strict subset of
recompressed: of the entries that had to be rebuilt, this many went out as
a computed delta rather than whole. A full clone measures all three as zero,
because a whole-repository request contains every base.
§The format
git’s delta stream, which crate::resolve already applies: the base’s size
and the target’s size as LEB128, then instructions. A byte with the top bit
set is a copy — the low nibble selects which of four offset bytes follow,
bits 4-6 which of three size bytes, all little-endian and all absent when
zero. A byte with the top bit clear is an insert of that many literal
bytes, 1..=127.
Two encoding hazards, both avoided by construction rather than by comment:
- a copy whose three size bytes are all zero means 65536, not zero, so a
zero-length copy must never be emitted — [
Delta::copy] is only ever called withlen >= MIN_MATCH; - a copy offset is four bytes, so a base at or past 4 GiB cannot be named.
deltarefuses such a base up front rather than truncating one.
§The search, and what it deliberately does not do
git pack-objects sorts every object by type, path hash and size and then
tries a sliding window of 10 candidates per object, at pack.depth 50. That
is a global search and it is the expensive half of packing.
This does none of it. The caller arrives with one candidate base — the nearest ancestor of the entry’s own stored delta chain that is inside the request — and this module only has to encode against it. The candidate is free: it is read out of the archive’s own back-references, which are the packer’s original similarity judgement, already made and already stored.
§What it produced, end to end
One server process, one seeded store, oden 2026-08-11, git fsck --full --strict exit 0 on every arm. The only difference between the columns is
[enabled]:
arm objects whole (off) re-delta (on)
full clone 36 172 6 200 252 6 200 252 unchanged
narrowed clone `base` 31 805 11 613 666 6 424 726 1.81x
--filter=blob:none 16 491 1 566 022 1 566 022 unchanged
incremental fetch 4 367 1 339 879 1 339 879 unchangedThe narrowed pack’s own entry shapes say where it went: 3 153 whole entries become 2 294, so exactly 859 of the 961 found an in-request ancestor — the number this module’s search predicted before it was written — and 5 188 940 bytes came off, 6 041 per entry. Stock git sends 5 820 000 for the same 31 805 objects, so what was 2.00× git is now 1.10×.
§Cost
One pass over the base to index it and one over the target to encode, both linear, with a 16-byte block hash and a bounded collision chain, plus one inflate of the base — which the single-slot cache spares for a run of entries sharing one ancestor.
Against that, the deflate it feeds is strictly cheaper than the one it
replaces: a ~1 KB delta instead of a 16 KiB object. The two roughly cancel.
Measured on the server’s own utime + stime over 10 narrowed clones, three
interleaved rounds, CPU pressure gated below 2.00: 139 ms per clone with
it on, 137 ms with it off, against a within-arm spread of 133-145. So the
1.81× on the wire costs nothing measurable in CPU, which is the trade this
engine had to be able to make and could not be assumed.