Module memguard 

The indexer must never take the box down.

MEASURED 2026-08-29 21:10 UTC on the gunnar appliance (16 GB, 4 vCPU, no swap, PID 1 = gunnar-server): a push --mirror of torvalds/linux (8.12 GiB pack, 13 948 244 objects, 11 215 872 deltas) was appended and acked, then the post-ack indexer grew past 16 GB and the kernel said Kernel panic - not syncing: System is deadlocked on memory. PID 1 was the only process, so the OOM killer had nothing to kill — and a kernel that cannot kill anything panics.

This module is the guard that makes that impossible by construction rather than by hoping the pack is small: before every batch and every few thousand objects inside one, the worker reads MemAvailable out of /proc/meminfo and, below a budget, flushes what it holds and waits. The kernel’s own estimate is used because it already accounts for reclaimable page cache — the mmapped pack ([crate::archive_map]) costs nothing against it, only the anonymous heap this process really owns does.

knob	default	what
ZNIPPY_GIT_MEM_BUDGET_PCT	20	pause when MemAvailable < this % of MemTotal
ZNIPPY_GIT_MEM_BUDGET_BYTES	—	absolute override of the same floor
ZNIPPY_GIT_MEMINFO	/proc/meminfo	the file read — a test hands in a fake

The pause is one WARN line when it starts (index.paused, which the gunnar dashboard renders as index paused: 2.9G available < 3.2G budget), a re-check every 250 ms, and one INFO line when it resumes. Nothing is logged per poll.

What “flush what it holds” means is the caller’s: MemWatchdog::pause_until_ok takes the flush as a closure, because the resolver’s droppable state (its held delta bases, crate::resolve) and the sink’s buffered rows are not this module’s to know about.

Structs

MemWatchdog

The watchdog: a meminfo path, a floor, and the counters that say what it did.

Pause

What one pause did — milliseconds, how it ended, and where memory stood.

Enums

PauseEnd

How a pause ended, which is the fact the caller has to branch on.

Constants

CHECK_EVERY_OBJECTS

Objects between two reads of /proc/meminfo inside one pack. A read is a few microseconds; 4096 objects of inflate+hash is milliseconds.

DEFAULT_BUDGET_PCT

The default floor: pause when less than this share of MemTotal is available.

GIVE_UP_AFTER_EMPTY_FLUSHES

A pause whose flush frees nothing this many times running gives up before the deadline. index.spilled fired 190 and 627 times with scanned_rows_dropped=0 on every single one: the spill had nothing to give, because the RAM was held by the resolver’s live O(pack) structures. More waiting cannot fix that, and pretending otherwise is what cost two hours.

PAUSE_DEADLINE

A pause has a deadline. MemWatchdog::pause_until_ok had none, which is why the two rig boxes parked until somebody deleted the repository. Past this the wait ends with PauseEnd::Deadline and the caller must act — proceed, shed, or fail the pack BY NAME — rather than sleep again.

POLL

Re-check cadence while paused.

REFLUSH_EVERY

While paused, the flush is run again this often: what a reader rebuilt in the meantime (its scanned pack, the exploded oid tree) is the owner’s to drop again. MEASURED on the 8 GB rig 2026-08-30: a pause that flushed once sat for two hours at 92 % used, holding a 2 GB table a reader built AFTER the flush.

RESUME_MARGIN_BYTES

Resume once this much MORE than the floor is available — an absolute margin, not a share of the budget.

Functions

fmt_bytes

1.5G / 512M / 4K — the dashboard’s spelling, so the WARN row and the header agree on what a number looks like.

global

The process-wide watchdog, built from the environment on first use. One per process because the floor is a property of the box, not of a repository. An Arc so a worker mid-pack keeps the one it started with if a test swaps the global underneath it.

mem_total

MemTotal in bytes, or 0 when unreadable. What byte budgets expressed as a share of the box are derived from.

parse_meminfo

(MemTotal, MemAvailable) in bytes out of /proc/meminfo text.

peak_rss_bytes

This process’s peak resident set (VmHWM off /proc/self/status) in bytes, or 0 when unreadable. The audit line carries it: on the appliance the server is PID 1, so its high-water mark is the box’s.

set_global

Replace the process-wide watchdog — what a test does to put a fake /proc/meminfo under a real store. Returns the previous one so the test can put it back.