ringfire 🔥

ringfire is an ultra-low-latency, zero-copy, lock-free Inter-Process Communication (IPC) ring buffer and shared memory state bus for Linux.

It is engineered for high-frequency trading (HFT) engines, real-time market data ingestion, telemetry buses, and performance-critical distributed pipelines where microsecond socket latencies and kernel overhead are unacceptable.

💡 The Problem: Why Traditional IPC Fails Under High Load

When communicating between processes on the same host, developers usually default to Unix Domain Sockets (UDS), TCP loopback, pipes, ZeroMQ, or broker-based message queues (NATS, Redis). In high-throughput, low-latency environments, these primitives introduce severe architectural bottlenecks:

The Three Critical Pain Points:

1. The Syscall & Context Switch Tax: Every write() and read() triggers CPU privilege elevation from user-space to kernel-space and back, polluting CPU L1/L2 caches and branch predictors.
2. Buffer Bloat & Head-of-Line Blocking: If a consumer process stalls (e.g. garbage collection pause in Python, disk IO hiccup, or debug pause), standard socket buffers fill up immediately, stalling the critical producer or blowing up memory.
3. Serialization Overhead: Marshalling data to and from JSON, Protobuf, or even compact binary encoders consumes valuable CPU cycles and allocates memory on the hot path.

🎯 The Solution & Vision: What ringfire Solves

ringfire moves the communication fabric directly into physical RAM via POSIX shared memory (/dev/shm):

• Pure Shared Memory (/dev/shm): Producer and consumers map the exact same physical memory region directly into their virtual address spaces.
• Atomic Acquire/Release Synchronization: State is coordinated via 64-bit atomic sequence numbers using CPU-level memory barriers (core::sync::atomic), completely bypassing the operating system kernel.
• Single-Writer Freedom (LatestWins Policy): The producer always writes to the ring. Slow, paused, or dead consumers can never block, stall, or crash the producer. If a consumer falls behind the ring buffer capacity, it detects that it was lapped and skips cleanly to the live stream.
• Cache-Line Isolated Layout: Memory structures are aligned to 128-byte cache lines to eliminate false sharing between producer write heads and consumer read heads.
• O(1) State Blackboard: Besides sequential stream events, ringfire provides a direct seqlock-synchronized slot table. Consumers can instantly inspect the latest state (e.g., current Best Bid & Offer for 500 coins) in ~5 nanoseconds without replaying historical events.
• Polyglot First-Class Support: Because the layout in /dev/shm is standard C-ABI memory, consumers can be written in Rust, C, C++, or Python (mmap + ctypes/numpy) with zero bridge penalty.

🤝 The Rapidfire Family

ringfire is designed as the cross-process counterpart to rapidfire:

┌─────────────────────────────────────────────────────────────────────────────┐
│                          IN-PROCESS (Single Process)                        │
│                                                                             │
│                                  rapidfire                                  │
│           • Intra-process MPMC / MPSC channels across threads & Tokio       │
│           • Ultra-low latency (< 15 ns), zero heap allocations              │
└──────────────────────────────────────┬──────────────────────────────────────┘
                                       │
                                       ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│                         CROSS-PROCESS (Multi-Process IPC)                   │
│                                                                             │
│                                  ringfire                                   │
│           • Inter-process lock-free ring buffer via /dev/shm                │
│           • O(1) Shared State Blackboard (Seqlock)                          │
│           • Sub-50ns latency, C11 header, Python zero-copy bindings         │
└─────────────────────────────────────────────────────────────────────────────┘

🚀 Verified Benchmarks (AMD Ryzen 9 7950X on Linux booster)

Benchmarked using Criterion directly against POSIX shared memory (/dev/shm) on host booster (16 Cores / 32 Threads, Linux 6.8):

📦 Quickstart (Rust)

1. Producer: Publish Fixed-Size Binary Records

use ringfire::RingProducer;

#[repr(C)]
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
struct MarketTicker {
    asset_id: u32,
    bid_px: u64,
    ask_px: u64,
    timestamp_ns: u64,
}

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Creates a 65,536 slot ring buffer in /dev/shm/hft_ticker_stream
    let mut producer = RingProducer::<MarketTicker>::create("/dev/shm/hft_ticker_stream", 65536)?;

    let ticker = MarketTicker {
        asset_id: 42,
        bid_px: 64_250_000_000,
        ask_px: 64_250_500_000,
        timestamp_ns: 1_726_870_000_000_000,
    };

    // Pushes ticker directly to shared memory (1.70 ns)
    producer.push(&ticker);

    Ok(())
}

2. Tokio Async Consumer: Cooperative Non-Blocking Streaming

In async bots, busy loops starve the Tokio runtime. AsyncRingConsumer solves this with adaptive fast-path spinning, cooperative tokio::task::yield_now(), and 0% CPU idle sleep:

use ringfire::AsyncRingConsumer;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let mut consumer = AsyncRingConsumer::<MarketTicker>::attach("/dev/shm/hft_ticker_stream")?;

    println!("Attached async consumer. Streaming market data...");

    loop {
        // Yields cooperatively to other Tokio tasks when no traffic is present
        let ticker = consumer.recv().await;
        // Process ticker without stalling the Tokio runtime
    }
}

3. Synchronous Low-Latency Consumer (Dedicated Cores)

use ringfire::{RingConsumer, BusySpin};

let mut consumer = RingConsumer::<MarketTicker>::attach("/dev/shm/hft_ticker_stream")?;
let mut wait = BusySpin::new();

loop {
    // Spin polling for sub-20ns reaction time
    let ticker = consumer.recv_blocking(&mut wait);
    // Process ticker
}

4. Shared State Blackboard (O(1) Snapshot Table)

use ringfire::{BlackboardProducer, BlackboardConsumer};

// Producer updates snapshot table
let mut bb_prod = BlackboardProducer::<MarketTicker>::create("/dev/shm/hft_state_table", 1024)?;
bb_prod.write(42, &ticker)?; // 1.19 ns write

// Consumer reads instantaneous current state by asset ID
let bb_cons = BlackboardConsumer::<MarketTicker>::attach("/dev/shm/hft_state_table")?;
if let Some(state) = bb_cons.read(42)? { // 4.09 ns O(1) tear-free read
    println!("Current BBO for asset 42: bid={}, ask={}", state.bid_px, state.ask_px);
}

5. Raw Binary Payloads ([u8; N]) & Pointer Casting

You can also operate over raw byte buffers without declaring fixed structs:

use ringfire::{RingProducer, AsyncRingConsumer};

// Producer sends a raw 64-byte binary packet
let mut producer = RingProducer::<[u8; 64]>::create("/dev/shm/raw_stream", 65536)?;
let raw_bytes = [0xAAu8; 64];
producer.push(&raw_bytes);

// Consumer reads the 64 bytes and casts to struct in-place
let mut consumer = AsyncRingConsumer::<[u8; 64]>::attach("/dev/shm/raw_stream")?;
let bytes: [u8; 64] = consumer.recv().await;
let ticker: &MarketTicker = unsafe { &*(bytes.as_ptr() as *const MarketTicker) };

🛡️ Memory Safety: Why Raw Pointers into Shared Memory Are Dangerous

In cross-process shared memory with a non-blocking writer (LatestWins), returning a raw pointer (*const T) directly into the mapped /dev/shm buffer is fundamentally unsafe:

• If a consumer holds a raw pointer to slot $K$, and the writer laps the buffer and begins overwriting slot $K$ on another CPU core, the consumer will observe a torn read (half old data, half new data).
• In high-frequency trading and order book streaming, a torn read corrupts prices and sizes, leading to disastrous trading errors.

The ringfire Solution: Two-Phase Seqlock Validation

ringfire enforces tear-free memory safety without locks:

1. Consumer loads slot sequence $s_1$ with Ordering::Acquire.
2. Copies the payload into the consumer's local registers/stack (takes 1 CPU clock cycle for 32–64B via vmovups).
3. Loads slot sequence $s_2$ with Ordering::Acquire.
4. If $s_1 == s_2$, the copy is mathematically guaranteed to be consistent, uncorrupted, and tear-free.
5. If $s_1 \neq s_2$ (writer updated the slot mid-read), the consumer immediately discards the partial copy and re-evaluates the latest sequence.

[!WARNING] Anti-Pattern: Returning Raw Pointers into Shared Memory (*const T) Some naive IPC designs attempt to return a direct pointer or slice &[u8] into /dev/shm to claim "zero-memcpy". In a multi-process architecture with a non-blocking writer (LatestWins), this is a dangerous anti-pattern: the writer can overwrite that memory slot at any microsecond while the reader is parsing it, causing undefined behavior, silent data races, and torn reads.

ringfire deliberately copies the slot payload into the reader's stack/register space inside a seqlock validation boundary (s1 == s2). For modern x86_64/ARM64 architectures, copying 32–64 bytes takes ~1 CPU clock cycle (via vmovups) and is orders of magnitude faster than recovering from corrupted state or dealing with UB.

⚡ Wait Strategies

ringfire supports selectable wait strategies depending on CPU budget:

• WaitStrategy::BusySpin: Sub-30ns reaction time. Spins tightly on CPU (core::hint::spin_loop()). Recommended for dedicated HFT cores.
• WaitStrategy::YieldBackoff: Spins for $K$ iterations then calls std::thread::yield_now(). Balanced CPU usage with ~150ns reaction time.
• WaitStrategy::Futex: Sleeps on Linux futex when the queue is idle. Consumes 0% CPU when waiting, wakes up via atomic event signaling.

🐍 Polyglot Access (C / C++ / Python)

• C11 Header: Include include/ringfire.h in any C/C++ project without linking overhead.
• Python (ringfire-py): Read shared memory slots directly via Python mmap and ctypes.Structure with no serialization penalty.

👥 Author

Alexander Panasenko

• Email: alex@prod.codes
• GitHub: @alex09x

📜 License

Licensed under either of:

• Apache License, Version 2.0 (LICENSE-APACHE)
• MIT License (LICENSE-MIT)

at your option.