Module offload

Expand description

A bounded, shared worker pool for blocking work (the M/M/c side of the hybrid design).

Bounded queue. OffloadPool::try_submit fails at once with Rejected::Full when max_queue jobs are waiting; nothing grows without bound.
Workers on demand. A worker thread is started only when a job arrives and no worker is idle, up to max_threads; idle workers exit after idle_timeout.
CoDel admission. Each job’s sojourn time (from submission until a worker takes it) is measured. When it stays above target for a whole interval, the pool reports overload and refuses new jobs with Rejected::Overloaded until a job is taken faster than target again (Nichols and Jacobson, “Controlling Queue Delay”, 2012). A server answers those with a fast 503 instead of queueing. The effective target is the larger of target and four times the jobs’ recent mean run time, so a pool of slow blocking jobs (100 ms calls to a database) does not shed because a job waits a few run times behind a burst: only a queue that keeps growing past what the workers can absorb is overload (K10: at the old one-run-time target, 500 clients of 100 ms jobs on 192 workers had nearly every request shed).

Structs§

OffloadConfig: Settings of an OffloadPool.
OffloadPool: A bounded shared worker pool. Cheap to clone. See the module docs.
OffloadStats: Counters of an OffloadPool.

default_threads: The worker cap for max_threads: 0: 32 per CPU, at least 512 and at most 1024. Sized for blocking integration calls like a thread-per-request server’s pool (a 100 ms handler needs about 10 threads per 100 requests per second; Cameleer’s pool is 1024); workers start on demand and exit after idle_timeout, so the cap costs nothing until jobs block (K10).