rapidfire

An ultra low-latency, lock-free, zero-dependency async MPMC channel for Rust, built for trading bots, WebSocket fan-in and other latency-critical pipelines where a lost message is unacceptable and every nanosecond on the hot path counts.

The core is an in-crate lock-free block queue; the crate depends on nothing but std.

Key features

• ⚡ Fastest path measured: 5 ns per push+pop and 5 ns per SPSC message on Zen 4; 18 ns per SPSC message on Neoverse-N1 (see the tables below).
• 🧱 Own lock-free block queue: 63-slot blocks, one CAS (or one wait-free fetch_add) per claim, per-slot state words tagged with the lap number, blocks recycled through a pool so steady-state traffic never touches the allocator.
• 🧭 Cache-line discipline (128 bytes): producers never read the consumers' line and consumers never read the producers' line; the only lines that cross cores per message are the slot lines carrying the values. Read marks live on a consumer-owned line.
• 💤 Zero-cost sleep, no SeqCst on the hot path: while messages flow nothing is locked and no waker is cloned; a producer does one Relaxed load after its claim. The party that parks pays with one RMW on the other side's index (release-sequence hand-off, model-checked with loom).
• 🔄 Cancellation safe: dropping a Recv/Send future (e.g. in tokio::select!) removes its waker; counters never drift.
• 🔌 Drop-in API: unbounded, bounded, send, recv, try_send, try_recv, close, len, counts — the same shape as async-channel.

Benchmarks

Two harnesses live in benches/: raw_bench.rs (synchronous, thread-based, pinned, the same code driving every implementation, u64 payloads) and real_case.rs (the message shapes and topologies of the trading bots this crate was written for, both as std threads and as tokio tasks). channel_benchmarks.rs adds criterion/tokio async groups. All numbers are medians of 5 runs after a warm-up; the full set for every machine and scenario, the perf profiles and the notes on the cells we lose are in benches/RESULTS.md.

Toolchain and library versions: rustc 1.97.1 (8bab26f4f 2026-07-14), -C target-cpu=native, lto = "fat", codegen-units = 1; compared against crossbeam-queue 0.3.14 (SegQueue, ArrayQueue), flume 0.11.1, async-channel 2.5.0, tokio 1.53.1 (sync::mpsc) and std::sync::mpsc of the same toolchain.

Machines

Raw harness, ns per message (lower is better)

SPSC 1→1, unbounded

SPSC 1→1, bounded(1024)

MPSC 4→1, unbounded

MPSC 4→1, bounded(1024)

MPMC 4→4, unbounded

MPSC 32→1, unbounded (market-data collectors: 10–40 WebSocket readers feeding one writer; 32 producer threads on 12–18 cores, so this is oversubscribed like production)

MPMC 4→4, bounded(1024)

Ping-pong round trip (two channels, two threads)

1 thread, push+pop pair

rapidfire versus the best other implementation in each cell (×, >1 means rapidfire is faster; the 8→1 rows and the burst row are in RESULTS.md)

Real bot topologies

AMD Ryzen 9 7900 (Zen 4), one 6-core CCD (40 readers: all 12 cores) — std threads, spinning (ns per message; round trip for command/response; lower is better)

AMD Ryzen 9 7900 (Zen 4), one 6-core CCD (40 readers: all 12 cores) — tokio tasks, 4 workers, async send/recv (ns per message; round trip for command/response; lower is better)

The same tables for the Ryzen 9 7950X, the Neoverse-N1 box and the M3 Pro are in RESULTS.md, together with the explanation of the cells where async-channel or ArrayQueue is ahead and what is planned for them.

How it works

src/queue.rs documents the protocol in full; the short version:

• Positions are absolute indices; a block serves one lap of 64 positions, the last of which is a sentinel that never holds a value and marks a block transition.
• A producer claims a position (wait-free fetch_add while no contention has been seen, CAS otherwise — with several producers CAS serialises the claims so adjacent-slot writes do not fight over one cache line, which is 4× faster than fetch_add on both Zen 4 and N1), locates its block by walking prev links back from tail.block, writes the value and publishes the slot with a Release store.
• A consumer looks at the head slot's lap-tagged state and claims it with a CAS only once it is written, so it never waits on a producer and the head never moves past an unwritten slot. The consumer that takes the last slot moves the head to the next block, waits for the other readers' marks and recycles the block.
• Blocks are never freed while the channel lives (a spare slot plus a pool), which is what makes stale block pointers safe to inspect. Memory stays at the channel's high-water mark until it is dropped.
• Sleeping: a receiver registers its waker, bumps a counter and does fetch_add(0) on the tail index; every later claim synchronises with it, so producers only need a Relaxed load of the counter after their claim. A slot that was claimed before the registration but not yet written makes the receiver poll briefly and then yield to its executor instead of sleeping.

Generated code

examples/asm_probe.rs wraps try_send/try_recv in never-inlined functions so the hot paths can be read with objdump. On x86-64 (Zen 4) an unbounded try_send is one lock xadd (or one lock cmpxchg once producer contention has been observed) plus about twenty ordinary instructions: the closed byte, the capacity word, the contention word, the tail block and its start position, the value store and a plain mov to publish the slot. try_recv is one lock cmpxchg plus about twenty instructions. On AArch64 the same paths are one LSE atomic (ldaddal/casal) with ldapr loads and one stlr. Everything that is not on that path (contended claims, the bounded capacity check, the block walk, the block transition) is out of line, so the inlined code at a call site stays small. Three alternative publish sequences were measured and rejected (dmb ishst + plain store, fence(Release) + plain store, and an RMW publish); they remain behind --cfg switches. perf profiles on Zen 4 and Neoverse-N1 (in RESULTS.md) show the remaining time in the locked claim and in the slot-line transfers between cores, not in the generated code. perf profiles on Zen 4 and Neoverse-N1 (in RESULTS.md) show the remaining time in the locked claim and in the slot-line transfers between cores, not in the generated code.

Verification

• cargo test — 10 unit, 46 integration (std threads and tokio: MPMC stress with every value received exactly once, bounded back-pressure across block boundaries, cancellation storms in select!, close while parked, Drop accounting, zero-sized and large payloads, waker re-registration, Debug/Display) and doc tests; also run with RUSTFLAGS="--cfg fcrs_small_blocks" (3-slot blocks) so every test crosses block boundaries and recycles blocks constantly. cargo llvm-cov reports 97.8 % of lines and 98.3 % of functions of the core covered (the rest is cold error handling).
• cargo test --release --features loom --test loom — loom model checks of the producer/consumer protocol, block transitions, close and the sleep/wake hand-off.
• cargo +nightly miri test --lib (with the small-block cfg) — no undefined behaviour in the unsafe core.
• GitHub Actions (.github/workflows/ci.yml) runs fmt, clippy, the test suite with both block sizes, loom and miri on x86-64 Linux, AArch64 Linux and macOS, plus a manual benchmark job (virtual machines: indicative numbers only).
• Multi-hour hang/crash hunts on the 128-core ARM box are what found and fixed the two bugs in the first drafts (a null prev on a recycled block; consumers passing an unwritten slot stranding a pre-empted producer's block lookup).
• An independent adversarial review of the memory-ordering arguments found two more, both now covered by regression tests: a Recv/Send future cancelled right after being woken swallowed the wake-up (it is now forwarded to the next parked waiter), and the block transition's index update was a plain store, which ends the release sequence a parked bounded sender relies on (it is an RMW now).

Known, documented race (shared with tokio's and async-channel's channels): a send that observed the channel open and then races with close() may succeed after the receivers already reported Closed; that value is dropped with the channel.

Quick start

[dependencies]
rapidfire = { git = "https://github.com/alex09x/rapidfire" }

use rapidfire::unbounded;

#[tokio::main]
async fn main() {
    let (tx, rx) = unbounded::<String>();

    tokio::spawn(async move {
        tx.send("BTC-USDT orderbook update".to_string()).await.unwrap();
    });

    if let Ok(msg) = rx.recv().await {
        println!("Received: {}", msg);
    }
}

License

Licensed under either of Apache License, Version 2.0 or MIT License at your option.