Crate burst_pool

This crate provides an SPMC channel for cases where values must be processed now or never, and workers are woken up all-at-once.

The intended use-case for this library is pretty specific:

• Most of the time things are quiet, but occasionally you have a lot of work to do.
• This work must be dispatched to worker threads for processing.
• Latency is important, but you have too many worker threads realistically spin-wait them all.
• When you receive work to do, you typically have more work than worker threads (ie. workers are woken all at once).

If the above does not apply to you, then the trade-offs made by this library probably aren't good ones.

Each successfully sent value is recieved by exactly one receiver. This crate is Linux-only.

Performance

The metric we care about is the latency between calling Sender::send() on the work-distributing thread, and Receiver::recv() returning on the threads which will process the work. We want it to be small and consistent. The design of burst-pool means that we expect this latency to be independent of the number of payloads sent. We also expect it to become much worse as soon as there are more worker threads than cores available.

I'm benchmarking burst-pool's performance against spmc, a crate which provides an SPMC channel with the normal queueing semantics when the pool is overloaded. The use-cases of spmc and burst-pool are quite different, but when the pool is not overloaded their semantics are the same.

Run cargo bench to make some measurements on your own machine. If your results are significantly worse than those below, your kernel might be powering down CPU cores too eagerly. If you care about latency more than battery life, consider setting max_cstate = 0.

2 payloads sent to 3 workers	1%	10%	50%	90%	99%	mean	stddev
burst_chan	3897	5144	8181	22370	31162	12146	31760
spmc	4260	5454	7279	14389	30019	9169	8484

3 payloads sent to 3 workers	1%	10%	50%	90%	99%	mean	stddev
burst_chan	4162	5116	8165	22000	35767	11565	14096
spmc	3911	5454	8895	23216	61595	12102	10197

5 payloads sent to 3 workers	1%	10%	50%	90%	99%	mean	stddev
burst_chan	3773	4786	8724	22877	34214	12091	10276
spmc	4241	5931	12542	1064532	1086586	432018	516410

6 payloads sent to 6 workers	1%	10%	50%	90%	99%	mean	stddev
burst_chan	5875	7344	12265	30397	47118	15984	10130
spmc	4170	7050	14561	34644	59763	18003	12511

In the 6/6 benchmark, the number of workers is greater than the number of cores on the benchmark machine.

Usage

// Create a handle for sending strings
let mut sender: Sender<String> = Sender::new();

// Create a handle for receiving strings, and pass it off to a worker thread.
// Repeat this step as necessary.
let mut receiver: Receiver<String> = sender.mk_receiver();
let th = std::thread::spawn(move ||
    loop {
        let x = receiver.recv().unwrap();
        println!("{}", x);
    }
);

// Give the worker thread some time to spawn
sleep_ms(10);

// Send a string to the worker and unblock it
sender.send(Box::new(String::from("hello")));
sender.wake_all();

// Wait for it to process the first string and send another
sleep_ms(10);
sender.send(Box::new(String::from("world!")));
sender.wake_all();

// Drop the send handle, signalling the worker to shutdown
sleep_ms(10);
std::mem::drop(sender);
th.join().unwrap_err();  // RecvError::Orphaned

Design

Each receiver has a "slot" which is either empty, blocked, or contains a pointer to some work. Whenever a receiver's slot is empty, it goes to sleep by polling an eventfd. When issuing work, the sender goes through the slots round-robin, placing work in the empty ones. It then signals the eventfd, waking up all sleeping receivers. If a receiver wakes up and finds work in its slot, it takes the work and blocks its slot. If a receivers wakes up and finds its slot is still empty, it goes back to sleep. When a receiver has finished processing the work, it unblocks its slot.

Structs

Receiver
Sender

Enums

RecvError