Expand description
Lock-free SPSC overwrite ring for high-rate telemetry in no-std contexts.
§Overview
- Single producer, single consumer.
- Producer never blocks; new writes overwrite the oldest slots when the ring wraps.
- Sequence numbers are monotonically increasing
u32;0is reserved to mean “empty”. - The consumer can drain in-order (
poll_one/poll_up_to) or sample the newest value (latest). - If the consumer lags by more than
N, it skips ahead and reports the number of dropped items. Two boundaries qualify that accounting: the sequence wrap can drop a few extra entries depending onN, and a gap of one whole sequence span aliases to “nothing new” and reports zero — see the two “Known limitation” sections below.
§Memory ordering
The producer invalidates the per-slot sequence, writes the value, publishes the new per-slot sequence, then publishes the newest sequence. The consumer validates the per-slot sequence before and after reading, which avoids observing a new value under an old sequence number when the producer overwrites a slot.
The barriers on both sides are fences rather than ordered accesses on the sequence itself: a
Release fence keeps the producer’s invalidation ahead of its value write, and an Acquire
fence keeps the consumer’s copy ahead of its re-check. Plain Release/Acquire on the
sequence stores and loads would leave the value access free to drift across the guard it is
supposed to be bracketed by.
Slot values are read and written with volatile accesses, and the consumer holds its copy as
MaybeUninit<T> until the re-check passes. A copy that raced with an overwrite is therefore
discarded as raw bytes and never materialises as a T that could violate the type’s validity
invariants — for reads that complete within one sequence span; the re-check compares sequence
values, so it carries the counter-width ABA bound stated under “Known limitation: whole-span
sequence aliasing” below.
§Known deviation: the seqlock data race
§What it is
This is a seqlock, and seqlocks are formally racy. The consumer may copy a slot while the
producer overwrites it; the sequence re-check then discards the copy. Miri’s data-race
detector reports that copy as undefined behaviour, and it is right to: read_volatile
constrains the compiler but does not make the access atomic.
§Why the design is this way
It is a deliberate trade, not an oversight, and the alternatives were rejected for reasons worth stating plainly:
- Make the producer wait for the consumer. This removes the race entirely, and removes the only property the type exists to provide. A telemetry producer in an interrupt handler cannot block on a consumer in a task loop.
- Copy the slot with atomic per-word operations. Sound, and unavailable: the word count has
to be computed from
size_of::<T>(), which needsgeneric_const_exprs(unstable). Falling back to per-byte atomics does not work either — anyTcarrying padding has uninitialised bytes even after a typed write, and an atomic load of uninitialised memory is itself UB. - Narrow the API so payloads live in atomics. A ring restricted to, say, a
u32oru64payload could store it in anAtomicU32/AtomicU64and would be fully race-free. This is a real option that was passed over in favour of accepting anyT: Copy. So the honest framing is that generality was chosen over formal soundness — not that Rust makes soundness impossible here.
§What this actually costs you
- Nothing is known to miscompile. Volatile seqlocks are used widely — the Linux kernel’s
seqlock_tis the same construct — and no compiler is known to break them. But “no known failure” is not a guarantee: the compiler is permitted to assume the race cannot happen.read_volatile/write_volatileblock the optimisations that would plausibly exploit it (splitting, duplicating, hoisting the copy); nothing blocks the ones nobody has thought of. - Loom shows it too, if you let it. A model that asserts delivered payload values fails: Loom serialises the non-atomic slot memory (the copy returns the latest bytes) while C11 coherence lets both Relaxed sequence checks keep returning the stale pre-invalidation sequence — a non-atomic read establishes no happens-before to force the re-check forward. That is the formal gap of the abstract machine witnessed concretely; the fence pairing above is what closes it on real hardware, where observing the new value implies the earlier invalidation is visible to the fenced re-check. The shipped models therefore assert the sequence protocol and conservation, never payload values.
- Your own Miri runs will flag it. If you run
cargo miri testover a test that drives this ring from two threads, you will get a UB report pointing into this crate. That is the deviation, not a new bug.scripts/miri.*shows the split-pass approach: full checking everywhere else, race detector off for this ring alone. - A raced copy is never returned — within the span bound. The double sequence check
discards it, and it is held as
MaybeUninit<T>until validated, so it cannot even briefly exist as aTthat violates the type’s validity invariants. The check compares sequence values, so a read preempted for one whole span of publications can pass both checks against a rewritten slot; see “Known limitation: whole-span sequence aliasing”.
§If that is not acceptable
crate::EventBufis race-free by construction — its producer and consumer never touch the same slot, and it passes Miri with the detector on. Note it is not a drop-in: it applies backpressure instead of overwriting, so a full buffer rejects the push rather than dropping the oldest entry. That is a different contract, and the right one only if your producer can handle failure.- If you need overwrite semantics and a clean Miri run, keep the payload out of the ring:
push a small index or handle into
crate::EventBuf, or into this ring accepting the caveat, and own the data elsewhere. - Keeping
Tsmall and padding-free does not remove the formal race, but it does remove any realistic tearing: a word-sized payload is copied by a single instruction on every target this crate supports.
§Known limitation: extra drops at the sequence wrap
Everywhere else these docs say the consumer keeps the last N entries and only loses data once
it lags by more than N. That holds for all but one moment in the ring’s life: the point where
the sequence counter wraps, once every 2^32 - 1 pushes.
Slots are addressed by (seq - 1) % N, but push skips the reserved value 0, so a full
cycle is 2^32 - 1 sequences rather than 2^32. Unless N divides 2^32 - 1, the slot walk
does not line up across the wrap: the index jumps instead of advancing by one, and for a window
straddling the wrap two live sequences can share a slot. The older of the two is overwritten
before the consumer had its full N entries of slack.
How much is lost depends entirely on N:
N | Entries lost, once per wrap |
|---|---|
| A power of two | Exactly 1 |
A divisor of 2^32 - 1 (3, 5, 15, 17, 51, 85, 255, 257, 65537, …) | 0 — the walk is seamless |
| Anything else | Up to N - 1; e.g. N = 48 loses 15, N = 96 loses 33, N = 121 loses 58 |
This is a data-loss bound, not a soundness problem. The affected read fails its sequence
check and is counted in PollStats::dropped, so read + dropped still accounts for every
published item and no stale or torn value is returned — both within the span bound of the
“Known limitation: whole-span sequence aliasing” section below, which is where each of those
guarantees runs out. It is indistinguishable from the ordinary lag-induced drops the consumer
already reports.
The same misalignment makes the lag-recovery jump resume up to one sequence later than it strictly needs to. That is bounded by the table above and reported identically.
Practical advice: prefer a power of two for N — the cost is one lost entry per 2^32
pushes, which is beneath the noise floor for any workload that also tolerates overwrite. Pick a
divisor of 2^32 - 1 if you want the wrap to be exactly seamless. Avoid values like 96 or 121
if a burst of drops at a predictable interval would matter to you. If no loss is acceptable at
all, crate::EventBuf applies backpressure instead and has no wrap boundary of this kind.
§Known limitation: whole-span sequence aliasing
Sequence arithmetic is modular. push skips the reserved value 0, so the counter cycles
through 2^32 - 1 distinct nonzero values, and every comparison and distance the consumer
computes is exact only up to that span. Two consequences follow — both inherent to any
fixed-width seqlock at its counter width:
- A whole-span gap from the resume cursor reports nothing. If the distance from the
consumer’s resume cursor to the newest publication reaches exactly
2^32 - 1(or any whole multiple), the published sequence aliases the cursor andpoll_one/poll_up_totake their nothing-new early return: zero reads and zero drops. Larger distances report only the remainder modulo the span. Theread + droppedconservation promise is therefore exact while the resume cursor stays within one span of the newest publication — residual backlog from a partial drain counts against that distance, so this is not simply “fewer than one span of publications between calls” (the sufficient call-cadence bound is below) — and silence after an extreme stall is not evidence that nothing was lost. - A whole-span mid-read stall defeats the sequence re-check. The torn-copy guard compares
the slot’s sequence before and after the copy. A consumer preempted inside that copy for
exactly one whole span of publications sees the same sequence value on both sides of a slot
that was rewritten in between — counter-width ABA — and a mixed copy would be accepted as
T. The discard argument for the documented deviation is therefore bounded: it holds for any read that completes in less than one full span of producer publications.
Reachability arithmetic, so the bound is a decision rather than a surprise: one span is
~4.29 billion publications. At a sustained 1 MHz push rate a poll gap must exceed ~71.6
minutes — and the mid-read stall must hold the consumer between two instructions of one
copy for that long — before either case is reachable; at 10 kHz it is ~5 days. The escape
hatch is structural, and the bound is measured from the resume cursor, not from call
cadence: aliasing needs the distance from the resume cursor to the newest publication to
reach one whole span, and a partial drain leaves residual backlog that counts against it. A
nonzero ordered poll (poll_one, or poll_up_to with a nonzero budget) always leaves the
cursor at most N - 1 behind the newest publication it observed at entry (each call freezes
that entry sample as its drain goal, which is also what bounds the call) — the lag-recovery jump
handles a lag over N, and draining even one item brings a lag of at most N below that —
so keeping the publications between consecutive nonzero polls below one span minus
N - 1 suffices. Consumer::skip_to_latest leaves the cursor exactly one behind the
newest it observed (so the next poll yields that newest item); its post-call allowance is
therefore one span minus one, not a full span.
poll_up_to(0, …) returns before touching the resume cursor, and the non-advancing
Consumer::latest never moves it. Separately, bound consumer preemption during a single
read to less than a span of publications. If neither bound can be stated for your system,
crate::EventBuf has no sequence wrap of any kind.
§Notes
TisCopyto allow returning values by copy without allocation.- The
&Tpassed to hooks is a reference to a local copy made during the read. - Sequence arithmetic goes through
seq_distance, which accounts for the reserved value0thatpushskips on wrap; raw wrapping subtraction over-counts by one across that boundary.
Structs§
- Consumer
- Consumer handle for reading from the ring.
- Poll
Stats - Outcome of a
Consumer::poll_up_toorConsumer::poll_onecall. - Producer
- Producer handle for writing into the ring.
- SeqRing
- Overwrite ring for SPSC high-rate telemetry. Producer never waits; consumer may drop if it lags > N.