Struct PpNRt 

pub struct PpNRt { /* private fields */ }

Implementations

impl PpNRt

pub fn get(e: &Engine) -> Result<&'static PpNRt, Box<dyn Error>>

The process-wide transport runtime, built on first use against the primary engine. The stage count + device map freeze at first build (one config per process — gates run one placement per invocation). Build errors are sticky and loud.

pub fn n_stages(&self) -> usize

pub fn acquire_walk( &'static self, path: &str, ) -> Result<PpWalkLease, Box<dyn Error>>

Acquire exclusive ownership of the PP boundary/event sequence for one complete model walk. Fail fast rather than blocking. A nested call can join only when its thread has an explicit coordinator borrow installed; merely running on the original thread is not authority.

pub fn cross_device(&self) -> bool

True iff any boundary crosses devices.

pub fn host_bounce_active(&self) -> bool

pub fn repeated_stage_device(&self) -> bool

Actual stage placement frozen when this runtime was built. Environment strings may be mutated by in-process gates later and are not authoritative for scheduler safety.

pub fn init_boundary_transport( &self, e: &Engine, n_embd: usize, ) -> Result<(), Box<dyn Error>>

Finish boot-time transport setup from the authoritative model width. This runs the production BoundarySlot ladder at 1/8/16/PRIME_CHUNK_MAX_TOKENS [n_embd] f32 rows once, then allocates host-bounce slots when selected. The loader calls it before uploading the first model weight; new_cache repeats the call as an idempotent guard before the first forward.

pub fn engine<'a>(&'a self, s: usize, primary: &'a Engine) -> &'a Engine

The engine a stage’s subgraph must run through: the primary engine when the stage lives on the primary device, else the stage’s own (remote-context) engine.

pub fn order_engine_behind( src: &Engine, dst: &Engine, ) -> Result<(), Box<dyn Error>>

Order dst’s OWN stream behind everything already enqueued on src’s OWN stream (memra#95).

This is NOT Self::fence_stages_behind and the difference is the whole bug that named it. A stage owns TWO streams: StageRt::stream, which only carries work issued inside an Self::enter scope (the ambient override), and — for every stage s > 0, including stages on the PRIMARY device, per the per-stage Engine isolation above — the stage’s own Engine’s stream, which is what rt.engine(s, e) launches on when the caller is NOT inside an enter scope. fence_stages_behind orders the first kind and says nothing about the second. Any body that hands a stage engine to a helper WITHOUT entering the stage (the glm5 spec round’s whole draft phase does exactly that, through glm5_head_engine) needs this one instead.

Same-Engine and same-stream calls are no-ops. Two Engines sharing a CUDA context (the deployed shape: the stage engines retain the same primary context) get an event record plus a stream wait, fully async. Genuinely different contexts (a stage on another device) fall back to draining src on the host, which is correct everywhere and costs one sync at session build, never per round.

TWO DELIBERATE ASYMMETRIES, both of which a “tidy it up” edit would get wrong:

• the SOURCE side reads src.stream(), the ambient-override-aware accessor, because the work being ordered was issued through the same accessor and must be the same stream even if a caller ever runs this inside a stage or enter_main scope. The DESTINATION side reads dst.gpu.main_stream(), override-blind, because the reader being ordered (the glm5 draft phase) provably never enters a scope, so its launches go to that Engine’s own stream and to nothing else.
• the context test is VALUE equality, not Arc::ptr_eq. CudaContext::new allocates a fresh Arc per call even though primary_ctx::retain hands back the same CUcontext, so every stage Engine on the primary device has a distinct Arc for the same context: Arc::ptr_eq would be false forever, the event path would be dead code, and every restored session would pay a host sync on the TTFT path this feature exists to protect (review round 2 on PR #100).

Parked on PpNRt rather than made a free function so it sits beside fence_stages_behind: the pair is the documentation.

pub fn bind_stage(&self, s: usize) -> Result<(), Box<dyn Error>>

Bind this OS thread to stage s’s CUDA context before issuing work there.

pub fn enter(&self, s: usize) -> StreamOverride

Enter stage s: until the guard drops, every engine op on this thread launches on the stage’s stream (memra_runtime ambient-stream override).

pub fn prepare_overlap_slots( &self, b: usize, n: usize, ) -> Result<(), Box<dyn Error>>

Allocate/grow BOTH slots for a boundary before pipelined issue starts. tx() can grow a slot lazily, but first-use ordering requires synchronizing the RX stream after that allocation. If slot 1 first grows after stage 1 of chunk N has already been queued, that sync drains chunk N and erases the only overlap in a two-chunk prime. Prewarming both slots pays the same one-time sync before either stage starts.

pub fn boundary_slot_growth_bytes( &self, b: usize, n: usize, ) -> Result<usize, Box<dyn Error>>

Project only the additional device bytes needed to grow this process-global boundary to n elements per slot. Admission must not charge the full persistent high-water to every session after it already exists.

pub fn tx( &self, b: usize, x: &CudaSlice<f32>, n: usize, ) -> Result<usize, Box<dyn Error>>

Boundary TX at boundary b (call within the stage-b scope; x = the materialized [n] residual): wait for the slot’s previous RX (write-after-read guard), copy x into the slot’s persistent buffer via the boundary’s transport on stage-b’s stream (the owning-stream/publication law), record ev_tx. Returns the slot index for the paired rx().

n is the PAYLOAD ELEMENT COUNT, not a fixed model constant: the eager arm passes n_embd (one row), the batched arm passes b_n * n_embd (B stacked rows, the [B, n_embd] boundary). The slot buffer is GROW-ONLY and the transport moves exactly the first n elements — batched serving changes B every tick (chunk fill), and a realloc-on-every-size-change would host-sync the RX stream per width change (see the SLOT FIRST-USE ORDERING note below for why each allocation needs that sync). Growing to the high-water mark makes the syncs O(distinct widths) instead of O(width changes).

pub fn tx_pipelined( &self, b: usize, x: &CudaSlice<f32>, n: usize, ) -> Result<usize, Box<dyn Error>>

Pipelined boundary TX: always alternate the shared double-buffer slots, independent of the decode-side MEMRA_PP_OVERLAP experiment flag. The boundary-local atomic keeps concurrent callers on one slot sequence rather than each restarting at A.

pub fn rx( &self, b: usize, slot_idx: usize, n: usize, ) -> Result<CudaSlice<f32>, Box<dyn Error>>

Boundary RX at boundary b (call within the stage-b+1 scope): wait on the slot’s ev_tx, copy the boundary buffer into a fresh working buffer (dtod on the RX stream — local on the RX device in both transports), record ev_rx. The returned buffer is RX-stage-owned: allocated, consumed, and eventually freed on that stage’s stream.

pub fn publish_to( &self, s: usize, dst: &Arc<CudaStream>, ) -> Result<(), Box<dyn Error>>

PUBLISH a DEVICE-RESIDENT result off the last stage to the caller’s stream (lane/pp2-spec 2026-08-06).

Every ppN body before this one returned HOST values — decode_step_h_ppn and decode_step_batch_ppn both dtoh inside the last-stage scope, and a dtoh on the producing stream is self-ordering. The verify trunk is the FIRST ppN body whose contract is device-resident output (decode_step_t_h_emb_dev exists precisely so the accept walk argmaxes on-device instead of moving T x n_vocab f32 per round), and device slices carry no stream affinity: the caller resumes on the PRIMARY stream and dereferences buffers whose producing kernels are still queued on the last stage’s stream. Nothing orders them.

Why this only ever failed on ONE device: with stages on separate devices the caller’s first touch is a cross-device copy that the driver orders against the source context, and the readback path syncs. Two streams on the SAME device genuinely overlap, so the primary stream reads a buffer whose matmul has not run — nondeterministic garbage (measured: NaN, 3155.677, and 2.87e-5 where the reference had -2.0048926), and it poisons the NEXT arm in the same process because the corrupted KV persists. This is the same class as the SLOT FIRST-USE ORDERING find above, one level up: there the unordered pair was alloc-memset vs TX copy, here it is stage-N compute vs the caller’s consumer.

Fix = the boundary law applied to the exit: record an event on the producing stage stream, make the caller’s stream wait on it. Event-wait, not a device sync, so the stage streams keep running for the deferred-readback arm. Call INSIDE the last-stage scope, after the last enqueue, with the caller’s (pre-enter) stream.

pub fn publish_all_to( &self, dst: &Arc<CudaStream>, ) -> Result<(), Box<dyn Error>>

PUBLISH EVERY STAGE to the caller (lane/glm5-accrace 2026-09-01) — the exit half of the boundary law, applied to ALL stages instead of only the producing one.

WHY publish_to(last, …) IS NOT ENOUGH. A ppN body’s terminal drain (a dtoh in the last-stage scope, or publish_to(n_st-1, …)) orders the caller behind the last stage, and the TX-wait chain transitively covers every earlier stage’s work UP TO its ev_tx. It does NOT cover what each earlier stage’s stream still holds AFTER its tx: the stage-scope locals (pos_d, the embedded/expanded rows, the boundary residual, every per-layer transient, and a verify round’s ckpt clones) are dropped with the stage override still active, so their free_async enqueues on the STAGE stream after ev_tx. The caller then resumes on its own stream and allocates — and, per the fence_stages_behind anatomy above, cudarc’s drop carries no read guard, so the pool can hand the caller a block whose stage-stream lifetime has not retired and the caller’s writes land under queued stage work.

MEASURED (research/glm53-flash-bringup-20260827/accrace-20260901/): with per-stage streams on one device, the hc ppN PRIME over a fixed 24-token prompt returned THREE distinct logit fingerprints within a single process — the first prime always canonical, later ones drifting — while MEMRA_PP_STREAMS=0 returned one fingerprint 11/11. Downstream, one glm5 spec round silently lost an acceptance (14/42 -> 13/42) and the e2e tape diverged.

Event waits, never a device sync: the stage streams keep running. Call at a ppN body’s EXIT with the pre-enter caller stream (a stage stream that IS the caller’s stream is skipped by publish_to).

MEMRA_PP_EXIT_PUBLISH=0 is the ROLLBACK SEAM (see pp_exit_publish) and restores the pre-lane, racy program exactly — it exists so the fix can be A/B’d in ONE binary and so a future perf question has a control arm, not because the guard is optional.

pub fn fence_stages_behind( &self, src: &Arc<CudaStream>, ) -> Result<(), Box<dyn Error>>

REVERSE PUBLICATION (#87 root cause, lane/pp2spec-crash 2026-08-07): order every STAGE stream behind the CALLER’s stream — the mirror of publish_to.

publish_to orders caller READS behind stage COMPUTE. Nothing ordered the other direction: buffers ALLOCATED on a stage stream (the verify’s returned logits/hidden, the VerifyCkpt stashes) are CONSUMED by kernels the caller enqueues on the PRIMARY stream, and when they drop, cudarc enqueues free_async on the ALLOCATING (stage) stream. With event tracking elided (the decode-path default) the drop carries no read-guard, so the pool can hand the block to the NEXT stage-stream allocation and its writes overwrite memory the queued primary-stream consumer has not read yet. Measured (research/pp2spec-crash-20260807): the spec round-seed read 13/4096 NaN = the uninitialized-bits signature (P(NaN|random u32) ~ 1/256), clean by host re-read time — a read-before-write race, fatal via the argmax-sentinel -> embed_gather MMU fault, and gated on c>=2 because a backed-up primary stream widens the window.

Fix law: before a ppN body enqueues NEW stage-stream work (allocations that may reuse freed blocks), every stage stream waits the caller’s stream at its current point. All primary consumers of the previous round’s stage-allocated buffers are enqueued by then (single host thread), so reuse-writes land strictly after them. Call at ppN-body ENTRY with the pre-enter caller stream. Door-shut configs never build a PpNRt, so single-card behavior is untouched.

pub fn record_done(&self) -> Result<CudaEvent, Box<dyn Error>>

Deferred readback: record a fresh completion event on the LAST stage’s stream (call after the step’s logits matmul has been enqueued there).

pub fn readback_stream(&self) -> &Arc<CudaStream>

The dedicated readback stream (last stage’s context).