Module queueing_theory 

Queueing Theory

Queueing theory is the mathematical study of waiting lines, and much of a delivery pipeline actually is a queue: a pull request waiting for a reviewer, a commit waiting for a CI runner, a ticket waiting to be picked up. Utilization — how busy a shared, capacity-constrained resource is, as a proportion of its available capacity — is the key quantity: wait time does not grow linearly with utilization, it grows sharply as utilization approaches full capacity.

Formula

Utilization       = arrival rate / service rate
Queue is stable    when utilization < 1.0

Why it matters

A resource running at 95% busy is often waiting many times longer than one running at 80%, not just “a little worse” — wait time grows sharply, not gradually, as utilization approaches capacity. A queue running at 100% utilization on average has effectively infinite wait time in practice, because real arrivals are uneven, not perfectly smooth. This is why “our reviewers are almost always busy” is a warning sign about wait times to come, not evidence of efficient resourcing, and why deliberate headroom below full utilization is a design choice, not waste.

Example

The topic doc’s own worked example: a shared CI fleet found running above 90% utilization during core hours — well past the point where queueing theory predicts wait time grows sharply rather than gradually. A queue with an arrival rate of 9 jobs/hour against a service rate of 10 jobs/hour runs at 90% utilization and is still stable; one with an arrival rate of 11 jobs/hour against the same service rate is not.

use software_engineering::queueing_theory::{utilization, is_queue_stable};

let near_capacity = utilization(9.0, 10.0).unwrap();
assert!((near_capacity - 0.9).abs() < 1e-9);
assert!(is_queue_stable(near_capacity));

let overloaded = utilization(11.0, 10.0).unwrap();
assert!(!is_queue_stable(overloaded));

Pitfalls

• Sizing a shared resource’s capacity to match its average arrival rate exactly: guarantees high utilization and runaway wait times whenever demand is even briefly uneven. Plan deliberate headroom.
• Reporting only mean wait time, never a percentile: hides the long tail that matters most to the people actually waiting in it.
• Blending success, failure, and skip into one throughput number: a team under pressure can make throughput look healthy by quietly letting the skip rate (abandoned or silently dropped work) rise. Track arrival, success, failure, and skip rate as four separate numbers.
• Treating “our people are always busy” as a compliment: it is a symptom of high utilization, the leading cause of long, unpredictable wait times.

Sources

• Chapter 2.7, “Queueing theory.”
• Little, John D. C. “A Proof for the Queuing Formula: L = λW.” Operations Research, 1961.

Topic doc: software-engineering-metrics/locales/en-001/chapters/02-07-queueing-theory.md

Functions

is_queue_stable

Whether a queue is stable: utilization strictly less than 1.0.

utilization

Utilization: arrival rate divided by service rate for a shared, capacity-constrained resource.