takoude
A lock-free single-producer single-consumer (SPSC) ring buffer optimized for low cross-core latency.
Usage
let = ;
spawn;
assert_eq!;
The capacity is a compile-time power of two and every slot is usable. The
two endpoints may move to different threads but cannot be cloned. push
and pop spin until space or data is available and can wait forever if
the other endpoint is dropped; try_push and try_pop return
immediately. try_push_with and try_pop_with access the slot in place.
Design
Each endpoint keeps its position in a local counter. Values are published through a sequence flag stored beside the payload, so a handoff costs one cache-line transfer and touches no shared head or tail atomics on the hot path.
The consumer releases capacity through a separate cache-padded progress counter and never writes to a consumed slot. The hot slot line therefore moves from producer to consumer only, instead of bouncing between them.
Feature flags
Both features are disabled by default. The default path is safe and portable.
| Feature | Description |
|---|---|
packed |
Single-atomic publication for AtomicValue payloads up to 4 bytes, via spsc_packed_channel. Safe, portable, Miri-checked. Halves the same-core cost on recent Intel cores. |
readahead |
The consumer prefetches 32 slots ahead on each pop. Large sustained-throughput gain on hosts with slow cross-core interconnects; benchmark on the target before enabling. |
Benchmarks
Results from current-generation AWS compute instances, with the benchmark pair pinned to physical cores on a stock Amazon Linux 2023 kernel. Latencies are P50. Every value is the median of three runs, each itself the median of three independently placed copies of the measurement kernel, because hot-loop placement alone can move single-run throughput by 30%.
c8a.xlarge, AMD EPYC 9R45 (Zen 5), cross-core floor near 80 ns RTT:
| Queue | Same-core P50 | Cross-core RTT/2 P50 | Throughput |
|---|---|---|---|
| takoude | 0.78 ns | 40 ns | 334 M msg/s |
takoude readahead |
1.02 ns | 30 ns | 287 M msg/s |
| rtrb | 1.48 ns | 40 ns | 526 M msg/s |
| heapless | 0.90 ns | 40 ns | 131 M msg/s |
| cyclos | 0.78 ns | 60 ns | 288 M msg/s |
c8i.xlarge, Intel Xeon 6975P-C (Granite Rapids). This instance exposes two physical cores about 210 ns RTT apart on the mesh, a much higher coherency floor that spreads the designs out:
| Queue | Same-core P50 | Cross-core RTT/2 P50 | Throughput |
|---|---|---|---|
| takoude | 1.9 ns | 101 ns | 36 M msg/s |
takoude readahead |
2.1 ns | 103 ns | 125 M msg/s |
| rtrb | 2.0 ns | 133 ns | 184 M msg/s |
| heapless | 0.95 ns | 160 ns | 203 M msg/s |
| cyclos | 1.2 ns | 209 ns | 146 M msg/s |
The c8i chart shows takoude built with packed and readahead, the
recommended configuration for hosts in this class: same-core from the
packed path, cross-core and throughput from the readahead build. The
other queues run their default configuration.
takoude trades sustained throughput for single-message latency. On the
slow Granite Rapids interconnect its handoff stayed between 99 and 106 ns
across every run, while rtrb ranged from 92 to 152 ns and cyclos sat near
double; on Zen 5 the cached-index queues all sit near the coherency floor
and takoude matches it at the lowest same-core cost. Queues that batch
shared-index updates reach higher sustained throughput; readahead
recovers most of that gap where the interconnect is slow (36 to
125 M msg/s on c8i) and is neutral to slightly negative on Zen 5.
For an older reference point: Intel Xeon Gold 6230 (Cascade Lake):
| Queue | Same-core P50 | Cross-core RTT/2 P50 | Throughput |
|---|---|---|---|
| takoude | 2.68 ns | 76 ns | 99 M msg/s |
takoude readahead |
2.7-2.9 ns | 77-80 ns | 157 M msg/s |
| rtrb | 4.77 ns | 116 ns | 153 M msg/s |
| heapless | 2.64 ns | 135 ns | 145 M msg/s |
| cyclos | 3.26 ns | 173 ns | 88 M msg/s |
The packed path on the cloud hosts:
| Host | Same-core P50, scalar | Same-core P50, packed |
Cross-core RTT/2 |
|---|---|---|---|
| c8a (Zen 5) | 1.0 ns | 0.9 ns | unchanged, 45 ns |
| c8i (Granite Rapids) | 1.9 ns | 1.0 ns | unchanged, 106 ns |
Cross-core handoff sits at the coherency floor on both hosts, so the publication encoding only moves the same-core cost.
Raw data: docs/results-c8a.csv,
docs/results-c8i.csv,
docs/results-c8i-readahead.csv, and
docs/results-clx.csv, collected with
docs/cloud-bench.sh.
cargo bench reproduces the suite on other hardware.
Verification
The test suite covers FIFO order, full and empty boundaries, sequence
wraparound, zero-copy paths, destruction, disconnect detection, and
cross-thread stress. The default and packed paths pass Miri. CI checks
formatting, Clippy, tests, and documentation.
The minimum supported Rust version is 1.88.
References
- L. Lamport. Specifying Concurrent Program Modules. TOPLAS, 1983.
- J. Giacomoni, T. Moseley, M. Vachharajani. FastForward for Efficient Pipeline Parallelism. PPoPP 2008.
- D. Vyukov. Bounded MPMC Queue.
- Intel. Faster Core-to-Core Communications.
- E. Rigtorp. Optimizing a Ring Buffer for Throughput.
- J. Wang et al. B-Queue: Efficient and Practical Queuing for Fast Core-to-Core Communication. IJPP 2013.
- J. Wang et al. BBQ: A Block-based Bounded Queue for Exchanging Data and Profiling. USENIX ATC 2022.
- N. Viennot. Core-to-Core Latency Dataset.
- Cyclos Research. Ring Buffers Research,
the source of the
cyclosentry in the comparison benchmark.
License
MIT