Skip to main content

Module resource

Module resource 

Source
Expand description

Staying out of the way on a loaded production server.

Priority, not thread starvation, is the primary lever: a nice’d worker uses idle capacity and is preempted the instant Apache wants the core. Measured separately on 610 MB of real archives: FADV_DONTNEED takes page-cache residency from 610 MB to 0 MB at no cost in wall time or CPU (33.68 s vs 34.14 s, within noise), where zgrep -a, rg -za and rg -za -j1 all leave the full 610 MB resident and evict whatever Apache and MariaDB had warm.

The platform asymmetry is real and is not papered over. FADV_DONTNEED evicts pages after reading; macOS has no posix_fadvise at all and its nearest equivalent, fcntl(fd, F_NOCACHE, 1), prevents caching before reading. So there are two hooks — prepare at open and finish after the archive has been read — and each platform does its real work in a different one:

LinuxmacOS
prepareFADV_SEQUENTIAL (readahead hint)F_NOCACHE (the actual work)
finishFADV_DONTNEED (the actual work)no-op

Functions§

finish
Called after an archive has been fully read.
load_average_1m
One-minute load average, from getloadavg(3): /proc/loadavg on Linux, the kernel’s own figure on macOS. None where there is none.
load_gate
Start adapting the concurrency to the load, or None when there is nothing to adapt: no --load-limit, a single job, or no load average to read.
next_gate
The gate after one more look at the load.
prepare
Called immediately after opening an archive, before it is read.
set_io_idle
Put our reads in the idle I/O class so they queue behind Apache’s.
set_priority
Lower our own scheduling priority. Unprivileged processes may always lower (never raise) their own, so this needs no capabilities and cannot fail in a way worth reporting.
starting_gate
The gate before any worker has started. All of the load is someone else’s at this point, so it is compared as it stands.