Struct AdaptiveRing 

pub struct AdaptiveRing { /* private fields */ }

Implementations

impl AdaptiveRing

pub const FRAME_INLINE_BUDGET: usize

Largest record stored inline in a ring slot by the frame path. Conservative across shapes: the smallest slot payload (Vyukov’s PAYLOAD_BYTES = 56) minus the 5-byte frame header (a class byte plus a u32 length), so an inlined record fits any shape’s slot no matter how the ring morphs.

pub const FRAME_DEFAULT_BLOCK_SIZE: usize = 8192

Block size of the lazily-created payload region. A frame larger than both the inline budget and this is rejected with RingError::PayloadTooLarge; size the region explicitly with with_frames for larger records.

pub fn create_anon( max_producers: usize, max_consumers: usize, capacity: usize, ) -> Result<Self, RingError>

Construct an adaptive ring with all backings pre-allocated.

max_producers and max_consumers size the composed MPSC + MPMC backings; runtime peer registration past these maxima is rejected. Initial shape is RingShape::Spsc.

pub fn create_hugepage( max_producers: usize, max_consumers: usize, capacity: usize, ) -> Result<Self, RingError>

Hugepage / large-page-backed adaptive ring (opt-in). Every backing (SPSC, each MPSC + MPMC producer ring, Vyukov) is laid out in its own huge / large page region instead of standard 4 KB pages, cutting TLB pressure for large rings: a 16 MB ring fits in a handful of 2 MB hugepages instead of thousands of 4 KB pages.

Cross-platform: Linux MAP_HUGETLB, Windows MEM_LARGE_PAGES, FreeBSD MAP_ALIGNED_SUPER, macOS x86_64 VM_FLAGS_SUPERPAGE_SIZE_2MB; only the per-backing region allocation is platform-gated, the compose-and-wire logic is shared with create_anon.

Requires a hugepage reservation (Linux vm.nr_hugepages) or the SeLockMemoryPrivilege (Windows); FreeBSD and macOS need no reservation (superpages are a transparent / on-demand hint, macOS x86_64 only). Returns Err when the backing cannot be allocated so the caller can fall back to create_anon.

pub fn create( path_prefix: impl AsRef<Path>, max_producers: usize, max_consumers: usize, capacity: usize, ) -> Result<Self, RingError>

File-backed adaptive ring. One file per backing (SPSC, each MPSC producer ring, each MPMC producer ring, Vyukov), named <path_prefix>.{role}.bin / <path_prefix>.mpsc.{i}.bin / <path_prefix>.mpmc.{i}.bin.

pub fn open( path_prefix: impl AsRef<Path>, max_producers: usize, max_consumers: usize, expected_capacity: usize, ) -> Result<Self, RingError>

Open an existing file-backed adaptive ring created by another process via AdaptiveRing::create with the same path_prefix + sizing. Validates each backing’s magic + capacity; does NOT re-initialize any layout, so in-flight items in the creator’s backings survive the attach.

The shape tag + pin generation are process-local: each process morphs / pins its own view. Cross-process callers coordinate the active shape out-of-band (or follow the creator’s sidecar) and call AdaptiveRing::morph_to to the agreed shape before pinning.

pub fn create_shmfs( name_prefix: &str, max_producers: usize, max_consumers: usize, capacity: usize, ) -> Result<Self, RingError>

Construct an AdaptiveRing whose four backings live in named RAM-resident shared memory regions (the ShmFs locale). Cross-process visible; never touches the page cache.

name_prefix becomes part of each backing’s logical shm name: {prefix}_spsc, {prefix}_mpsc_{i}, {prefix}_mpmc_{i}, {prefix}_vyukov. The same prefix on another process resolves to the same shared memory.

pub fn open_shmfs( name_prefix: &str, max_producers: usize, max_consumers: usize, expected_capacity: usize, ) -> Result<Self, RingError>

Attach to an AdaptiveRing whose four backings already live in the named shared-memory regions a different process created with create_shmfs.

The critical difference from create_shmfs: this validates each backing’s magic and attaches WITHOUT re-initialising the layout, so a snapshot the creator already enqueued survives the attach. (create_shmfs unconditionally re-lays-out every backing, which zeroes any data already in the region - correct for the creator, data-loss for a late attacher.) Use create_shmfs in the process that owns the region’s lifetime and open_shmfs in every process that joins it afterwards.

The peer directory is the source of truth for how many per-producer backings exist right now; max_producers / max_consumers are pre-attach floor hints only. Returns RingError::LayoutMismatch if a backing is absent or its header magic / capacity does not match (e.g. the creator has not run yet, or ran with a different capacity).

pub fn with_ordering_stamps(self) -> Result<Self, RingError>

Attach the ordering substrate: every subsequent push carries an 8-byte stamp in slot bytes [0..8) and the payload cap drops to STAMPED_PAYLOAD_BYTES (56 - the same 8 bytes Vyukov spends on its per-slot sequence atom). Pops through try_recv (and the pinned ordered_try_pop) strip the stamp and hand back payload bytes only.

Stamping is FIXED at construction - call this before any traffic. The merge flag inside the ordering region stays runtime-dynamic via set_ordering_mode.

Stamp-kind selection: invariant-TSC rdtsc when the CPUID probe passes, the shared counter on x86 without an invariant TSC, the monotonic clock on non-x86 hosts. Rings opened with AdaptiveRing::open adopt the creator’s stamp kind from the region header (validated, never re-initialised).

A stamped ring never morphs to RingShape::Vyukov: the stamped 64-byte slot layout does not fit Vyukov’s 56-byte slots, and the GlobalFifo declaration on a stamped ring is served by the merge flag instead of the Vyukov morph.

pub fn with_ordering_stamps_kind( self, kind: StampKind, ) -> Result<Self, RingError>

As with_ordering_stamps with an explicit stamp kind. StampKind::SharedCounter is the exactness opt-in: stamps form a total order at the price of one contended fetch_add per push. Opening an existing region with a kind that does not match the creator’s returns RingError::LayoutMismatch.

pub fn current_shape(&self) -> RingShape

Current shape.

pub fn peek_spsc_slot(&self) -> Option<PeekedSpscSlot<'_>>

Peek the next slot of the internal SPSC backing without copying or releasing. Returns None when the active shape is not SPSC OR when the ring is empty. Used by zero-copy egress paths (e.g. the bridge primitives’ write_all flow) when the active shape supports peek-direct.

The returned PeekedSpscSlot derefs to &[u8] pointing INTO the SPSC backing’s mmap region. Caller passes that slice straight to downstream consumers, then calls PeekedSpscSlot::confirm to release the slot.

pub fn is_empty(&self) -> bool

Shape-aware emptiness check across every backing this ring currently uses.

• SPSC: the single SPSC backing’s head==tail.
• MPSC: every per-producer SPSC sub-ring is empty.
• MPMC: every per-producer SPSC sub-ring in the grid is empty (cross-consumer claims are committed by sub-ring pops, so an empty grid means every slot has been consumed).
• Vyukov: producer_seq == consumer_seq.

Used by capacity-morph wrappers to decide whether a stale backing can be dropped. Conservative: a value returning true is guaranteed empty at the moment of observation across all sub-rings; concurrent producers writing into the active shape during the check cannot affect a stale-only caller because producers only target whichever Arc the wrapper’s ArcSwap currently points at.

pub fn approx_len(&self) -> usize

Shape-aware approximate item count across every backing currently in use (sum for composed shapes; single ring for SPSC / Vyukov). Used by sidecar policies to compute fill ratio and decide whether to grow / shrink capacity.

pub fn sub_ring_capacity(&self) -> usize

Capacity of a single underlying sub-ring (per-producer slot count). Composed shapes have N or N*M such sub-rings; the total slot inventory is sub_ring_capacity() * n_sub_rings. For SPSC / Vyukov this is the ring’s full capacity.

pub fn total_slot_capacity(&self) -> usize

Total slot inventory across every sub-ring this AdaptiveRing currently owns. For SPSC / Vyukov this is the same as sub_ring_capacity(). For MPSC / MPMC it is sub_ring_capacity() * n_sub_rings.

pub fn pin_generation(&self) -> u64

Current pin generation. Pinned handles capture this at pin time; a non-equal current value means the pin is stale.

pub fn max_producers(&self) -> usize

Number of per-producer backings this ring pre-allocated at construction. A HINT, not a ceiling: registration past it grows the backings on demand.

pub fn max_consumers(&self) -> usize

Consumer-count hint captured at construction. Consumer slots are claimed dynamically up to the substrate ceiling.

pub fn published_producers(&self) -> usize

Per-producer backings currently published (pre-allocated + grown), shared across every attached process.

pub fn contract(&self) -> RingContract

The ring’s effective contract. UNBOUNDED unless the caller declared one via with_contract - a declared contract is the ONLY thing that makes registration fallible; the default grows on demand.

pub fn with_contract(self, contract: RingContract) -> Self

Declare an explicit ring contract (builder; consumes self, like with_ordering_stamps). The contract’s count bounds become the attach-time admission check and its ordering / capacity constraints become the feasible-region filter a policy consults.

pub fn contract_filtered_shape(&self, target: RingShape) -> RingShape

Map a policy’s proposed shape to the nearest contract-legal one, so an auto-morph cannot violate the declared ordering contract by construction. A Fifo contract forbids the partitioned per-producer-lane shapes (Mpsc, Mpmc, which interleave producers); the order-preserving substitute is Vyukov on an unstamped ring. A stamped ring keeps the proposed shape - its global order is served by the MergeStrict flag, not a Vyukov morph (whose 56-byte slots do not fit the stamped 64-byte layout). Under the default (unbounded) contract this is the identity, so non-declaring rings are unaffected.

pub fn pin_shape(&self)

Pin the composed shape: stop the automatic reshape-on-register so the ring holds whatever shape it currently has. The user override for callers that want a fixed shape. An explicit morph_to pins implicitly.

pub fn resume_auto_shape(&self)

Resume the automatic shape (undo pin_shape / an explicit morph) and re-track the live peer counts. Unlike the automatic reshape - which never disturbs a Vyukov shape - this explicit resume DOES morph a Vyukov ring back to the counts-based composed shape (that is what resuming means).

pub fn shape_is_auto(&self) -> bool

Whether the composed shape auto-morphs to the active peer counts (the default). false after pin_shape or an explicit morph_to.

pub fn register_producer(&self) -> Result<usize, AdaptiveError>

Register a new producer. Returns its producer_id - a shared slot claim visible to every attached process. Registration GROWS the ring on demand (new per-producer backings past the construction hint) and auto-morphs the composed shape to the new peer counts; it fails only under a caller-declared contract ceiling (with_contract) or at the substrate slot ceiling (PRODUCER_SLOT_CEILING CONCURRENT producers). The id stays valid until unregister_producer.

pub fn unregister_producer(&self, producer_id: usize)

Unregister a producer slot. Caller passes the id returned from register_producer. The slot recycles; its backing (and any undrained backlog) stays until the consumer drains it.

pub fn register_consumer(&self) -> Result<usize, AdaptiveError>

Register a new consumer. Returns its consumer_id - a shared slot claim visible to every attached process. Rebalances MPMC ring ownership toward the new consumer set and auto-morphs the shape. Fails only under a caller-declared contract ceiling or at the substrate consumer-slot ceiling (CONSUMER_SLOT_CEILING).

pub fn unregister_consumer(&self, consumer_id: usize)

Unregister a consumer slot. The leaving consumer transfers its MPMC ring ownership to the remaining consumers itself (it is the single owner, so the direct transfer is safe), then releases the slot.

pub fn active_producers(&self) -> usize

Current active producer count (shared across processes).

pub fn active_consumers(&self) -> usize

Current active consumer count (shared across processes).

pub fn is_stamped(&self) -> bool

Whether this ring carries ordering stamps.

pub fn stamp_kind(&self) -> Option<StampKind>

Stamp kind, when stamped.

pub fn ordering_mode(&self) -> Option<OrderingMode>

Current ordering mode, when stamped. The mode atom lives in the MMF-resident ordering region, so every process attached to the ring reads the same value - deliberately unlike the process-local shape tag.

pub fn set_ordering_mode(&self, mode: OrderingMode) -> Result<(), RingError>

Flip the ordering mode. The ordered switch is one Release store: Off->On retroactively orders the in-flight backlog (stamps were already in the slots), On->Off is immediate. No drain, no data movement, and outstanding pins stay valid - the pinned pop consults the mode atom on every call.

pub fn inversions(&self) -> u64

Cross-producer inversions observed at pop since the ordering region was created. Shared across processes.

pub fn refresh_watermark(&self, producer_id: usize) -> Result<(), RingError>

Watermark heartbeat for an idle producer (MergeStrict liveness). See OrderingRegion::refresh_watermark.

pub fn retire_producer(&self, producer_id: usize) -> Result<(), RingError>

Terminal producer retirement: MergeStrict consumers stop waiting on this producer slot’s silence permanently. Call on clean producer exit; the slot must not push afterwards. See OrderingRegion::retire_producer.

pub fn release_drainer(&self, consumer_id: usize) -> Result<bool, RingError>

Voluntarily release the merge-drainer lease held by this process + consumer slot. Returns Ok(false) when the lease was not held.

pub fn tick_drainer_epoch(&self) -> Result<u64, RingError>

Advance the drainer-lease epoch (dead-drainer takeover after DRAINER_GRACE_EPOCHS missed beats). The QoS-aware sidecar ticks this once per scan; standalone callers tick it themselves, mirroring OwnerLease::tick_epoch.

pub fn ordering_region(&self) -> Option<&OrderingRegion>

Direct access to the ordering region for composing wrappers: capacity morphs seed the fresh backing’s region from the old one so counter stamps stay monotone across the swap, and E2E harnesses read watermarks / the drainer token directly.

pub fn try_send( &self, producer_id: usize, payload: &[u8], ) -> Result<(), RingError>

Adaptive-path push. One Acquire load on the shape tag, one branch, then the native push on the matching backend.

producer_id selects the producer ring for MPSC / MPMC shapes. For SPSC and Vyukov shapes the id is ignored (except on stamped rings, where it selects the producer’s stamp line and must stay below max_producers).

On stamped rings the payload cap is STAMPED_PAYLOAD_BYTES and the stamp is prepended transparently; the matching try_recv strips it.

pub fn try_recv( &self, consumer_id: usize, out: &mut [u8], ) -> Result<usize, RingError>

Adaptive-path pop. consumer_id selects the consumer’s round-robin partition for the MPMC shape. For SPSC, MPSC, and Vyukov shapes the id is ignored (one consumer).

On stamped rings this is the ordering-aware pop: the stamp is stripped (callers see payload bytes only, Ok(56)), the inversion counter runs, and when the ordering mode is MergeByStamp / MergeStrict the pop k-way-merges ring heads by stamp under the single-drainer lease.

pub fn with_frames(self, block_size: usize, block_count: usize) -> Self

Pre-create and size the frame payload region. Optional: the region is otherwise created lazily at FRAME_DEFAULT_BLOCK_SIZE the first time a record is too large to inline. No-op if the region already exists. Returns the ring for chaining.

pub fn send_frame( &self, producer_id: usize, payload: &[u8], ) -> Result<FrameClass, RingError>

Frame-path send: carries any payload size on whatever shape the ring is in. Records up to FRAME_INLINE_BUDGET go inline in the ring slot; larger ones spill to the shared payload region and the slot carries the block index. Returns which path the record took. producer_id selects the backing ring for MPSC / MPMC exactly as try_send. The same call works at every shape because the descriptor rides the slot and the region is multi-producer / multi-consumer safe.

Not available on stamped (ordering) rings - frames and stamps both claim the slot head, so they are mutually exclusive; returns RingError::LayoutMismatch there.

pub fn send_frame_as( &self, producer_id: usize, payload: &[u8], hint: LayoutHint, ) -> Result<FrameClass, RingError>

send_frame with an explicit layout override (LayoutHint::ForceInline / LayoutHint::ForceOffset).

pub fn recv_frame( &self, consumer_id: usize, out: &mut Vec<u8>, ) -> Result<FrameClass, RingError>

Frame-path recv: counterpart to send_frame. Clears out and fills it with the record’s payload, transparently reading the payload region and freeing its block for offset records. Returns which path the record took. consumer_id selects the consumer partition for MPMC as try_recv. Not available on stamped rings.

pub fn try_recv_with_stamp( &self, consumer_id: usize, out: &mut [u8], ) -> Result<(usize, u64), RingError>

As try_recv on a stamped ring, also returning the popped item’s stamp. This is how consumers assert the ordering guarantee they paid for (monotone stamps under the merge modes) instead of trusting it. Returns RingError::NotStamped on unstamped rings.

pub fn pin_current_shape(&self) -> PinnedRing<'_>

Pin the current shape and return a PinnedRing that exposes the matching backend at native speed. The composed arrays are captured at pin time (zero per-op indirection); producer growth bumps the pin generation, so pin holders see PinnedRing::is_still_valid == false and re-pin to pick up new backings.

pub fn morph_to(&self, new_shape: RingShape) -> Result<(), RingError>

Trigger a shape morph. NO data moves: the old shape’s backing becomes the STALE backing, producers follow the new shape_tag immediately, and the consumer’s pop path drains the stale backlog first (the stale walk) before reading from the new shape. This is what makes morphing safe under saturating traffic - there is no transfer to overflow the target’s capacity and no second drainer racing the live consumer (each backing keeps exactly one reader).

The stale marker stays set until the NEXT morph, which requires the backlog drained (RingError::StaleBacklog otherwise - the sidecar’s scan loop simply retries). Keeping it set gives a producer whose push straddled the tag flip a wide grace window: its item lands in the old backing, which the consumer still walks.

Bumps pin_generation so outstanding pins see is_still_valid() == false and re-acquire. Pinned NATIVE pops (spsc_try_pop etc.) are shape-direct and do not walk the stale backing; consumers that pop through pins across morphs use AdaptiveRing::try_recv or PinnedRing::ordered_try_pop, which do.

An explicit morph_to is a USER shape decision, so it pins the shape (suppresses the automatic count-driven reshape) until resume_auto_shape.

Trait Implementations

impl AdaptiveInstance for AdaptiveRing

fn header(&self) -> &HandshakeHeader

fn ring(&self) -> &ObservationRing

fn make_policy(&self) -> Box<dyn Policy>

fn apply_migration(&self, new_tag: u32)

Called by the sidecar when the policy returns a new strategy tag. Default implementation: just set the tag on the header. Primitives that need heavier migration (data-layout swap) override this to perform the swap before (or after) updating the tag.

impl Send for AdaptiveRing

impl Sync for AdaptiveRing