Struct PpNRt 

pub struct PpNRt { /* private fields */ }

Implementations

impl PpNRt

pub fn get(e: &Engine) -> Result<&'static PpNRt, Box<dyn Error>>

The process-wide transport runtime, built on first use against the primary engine. The stage count + device map freeze at first build (one config per process — gates run one placement per invocation). Build errors are sticky and loud.

pub fn n_stages(&self) -> usize

pub fn acquire_walk( &'static self, path: &str, ) -> Result<PpWalkLease, Box<dyn Error>>

Acquire exclusive ownership of the PP boundary/event sequence for one complete model walk. Fail fast rather than blocking. A nested call can join only when its thread has an explicit coordinator borrow installed; merely running on the original thread is not authority.

pub fn cross_device(&self) -> bool

True iff any boundary crosses devices.

pub fn host_bounce_active(&self) -> bool

pub fn repeated_stage_device(&self) -> bool

Actual stage placement frozen when this runtime was built. Environment strings may be mutated by in-process gates later and are not authoritative for scheduler safety.

pub fn init_boundary_transport( &self, e: &Engine, n_embd: usize, ) -> Result<(), Box<dyn Error>>

Finish boot-time transport setup from the authoritative model width. This runs the production BoundarySlot ladder at 1/8/16/PRIME_CHUNK_MAX_TOKENS [n_embd] f32 rows once, then allocates host-bounce slots when selected. The loader calls it before uploading the first model weight; new_cache repeats the call as an idempotent guard before the first forward.

pub fn engine<'a>(&'a self, s: usize, primary: &'a Engine) -> &'a Engine

The engine a stage’s subgraph must run through: the primary engine when the stage lives on the primary device, else the stage’s own (remote-context) engine.

pub fn bind_stage(&self, s: usize) -> Result<(), Box<dyn Error>>

Bind this OS thread to stage s’s CUDA context before issuing work there.

pub fn enter(&self, s: usize) -> StreamOverride

Enter stage s: until the guard drops, every engine op on this thread launches on the stage’s stream (memra_runtime ambient-stream override).

pub fn prepare_overlap_slots( &self, b: usize, n: usize, ) -> Result<(), Box<dyn Error>>

Allocate/grow BOTH slots for a boundary before pipelined issue starts. tx() can grow a slot lazily, but first-use ordering requires synchronizing the RX stream after that allocation. If slot 1 first grows after stage 1 of chunk N has already been queued, that sync drains chunk N and erases the only overlap in a two-chunk prime. Prewarming both slots pays the same one-time sync before either stage starts.

pub fn boundary_slot_growth_bytes( &self, b: usize, n: usize, ) -> Result<usize, Box<dyn Error>>

Project only the additional device bytes needed to grow this process-global boundary to n elements per slot. Admission must not charge the full persistent high-water to every session after it already exists.

pub fn tx( &self, b: usize, x: &CudaSlice<f32>, n: usize, ) -> Result<usize, Box<dyn Error>>

Boundary TX at boundary b (call within the stage-b scope; x = the materialized [n] residual): wait for the slot’s previous RX (write-after-read guard), copy x into the slot’s persistent buffer via the boundary’s transport on stage-b’s stream (the owning-stream/publication law), record ev_tx. Returns the slot index for the paired rx().

n is the PAYLOAD ELEMENT COUNT, not a fixed model constant: the eager arm passes n_embd (one row), the batched arm passes b_n * n_embd (B stacked rows, the [B, n_embd] boundary). The slot buffer is GROW-ONLY and the transport moves exactly the first n elements — batched serving changes B every tick (chunk fill), and a realloc-on-every-size-change would host-sync the RX stream per width change (see the SLOT FIRST-USE ORDERING note below for why each allocation needs that sync). Growing to the high-water mark makes the syncs O(distinct widths) instead of O(width changes).

pub fn tx_pipelined( &self, b: usize, x: &CudaSlice<f32>, n: usize, ) -> Result<usize, Box<dyn Error>>

Pipelined boundary TX: always alternate the shared double-buffer slots, independent of the decode-side MEMRA_PP_OVERLAP experiment flag. The boundary-local atomic keeps concurrent callers on one slot sequence rather than each restarting at A.

pub fn rx( &self, b: usize, slot_idx: usize, n: usize, ) -> Result<CudaSlice<f32>, Box<dyn Error>>

Boundary RX at boundary b (call within the stage-b+1 scope): wait on the slot’s ev_tx, copy the boundary buffer into a fresh working buffer (dtod on the RX stream — local on the RX device in both transports), record ev_rx. The returned buffer is RX-stage-owned: allocated, consumed, and eventually freed on that stage’s stream.

pub fn publish_to( &self, s: usize, dst: &Arc<CudaStream>, ) -> Result<(), Box<dyn Error>>

PUBLISH a DEVICE-RESIDENT result off the last stage to the caller’s stream (lane/pp2-spec 2026-08-06).

Every ppN body before this one returned HOST values — decode_step_h_ppn and decode_step_batch_ppn both dtoh inside the last-stage scope, and a dtoh on the producing stream is self-ordering. The verify trunk is the FIRST ppN body whose contract is device-resident output (decode_step_t_h_emb_dev exists precisely so the accept walk argmaxes on-device instead of moving T x n_vocab f32 per round), and device slices carry no stream affinity: the caller resumes on the PRIMARY stream and dereferences buffers whose producing kernels are still queued on the last stage’s stream. Nothing orders them.

Why this only ever failed on ONE device: with stages on separate devices the caller’s first touch is a cross-device copy that the driver orders against the source context, and the readback path syncs. Two streams on the SAME device genuinely overlap, so the primary stream reads a buffer whose matmul has not run — nondeterministic garbage (measured: NaN, 3155.677, and 2.87e-5 where the reference had -2.0048926), and it poisons the NEXT arm in the same process because the corrupted KV persists. This is the same class as the SLOT FIRST-USE ORDERING find above, one level up: there the unordered pair was alloc-memset vs TX copy, here it is stage-N compute vs the caller’s consumer.

Fix = the boundary law applied to the exit: record an event on the producing stage stream, make the caller’s stream wait on it. Event-wait, not a device sync, so the stage streams keep running for the deferred-readback arm. Call INSIDE the last-stage scope, after the last enqueue, with the caller’s (pre-enter) stream.

pub fn publish_all_to( &self, dst: &Arc<CudaStream>, ) -> Result<(), Box<dyn Error>>

PUBLISH EVERY STAGE to the caller (lane/glm5-accrace 2026-09-01) — the exit half of the boundary law, applied to ALL stages instead of only the producing one.

WHY publish_to(last, …) IS NOT ENOUGH. A ppN body’s terminal drain (a dtoh in the last-stage scope, or publish_to(n_st-1, …)) orders the caller behind the last stage, and the TX-wait chain transitively covers every earlier stage’s work UP TO its ev_tx. It does NOT cover what each earlier stage’s stream still holds AFTER its tx: the stage-scope locals (pos_d, the embedded/expanded rows, the boundary residual, every per-layer transient, and a verify round’s ckpt clones) are dropped with the stage override still active, so their free_async enqueues on the STAGE stream after ev_tx. The caller then resumes on its own stream and allocates — and, per the fence_stages_behind anatomy above, cudarc’s drop carries no read guard, so the pool can hand the caller a block whose stage-stream lifetime has not retired and the caller’s writes land under queued stage work.

MEASURED (research/glm53-flash-bringup-20260827/accrace-20260901/): with per-stage streams on one device, the hc ppN PRIME over a fixed 24-token prompt returned THREE distinct logit fingerprints within a single process — the first prime always canonical, later ones drifting — while MEMRA_PP_STREAMS=0 returned one fingerprint 11/11. Downstream, one glm5 spec round silently lost an acceptance (14/42 -> 13/42) and the e2e tape diverged.

Event waits, never a device sync: the stage streams keep running. Call at a ppN body’s EXIT with the pre-enter caller stream (a stage stream that IS the caller’s stream is skipped by publish_to).

MEMRA_PP_EXIT_PUBLISH=0 is the ROLLBACK SEAM (see pp_exit_publish) and restores the pre-lane, racy program exactly — it exists so the fix can be A/B’d in ONE binary and so a future perf question has a control arm, not because the guard is optional.

pub fn fence_stages_behind( &self, src: &Arc<CudaStream>, ) -> Result<(), Box<dyn Error>>

REVERSE PUBLICATION (#87 root cause, lane/pp2spec-crash 2026-08-07): order every STAGE stream behind the CALLER’s stream — the mirror of publish_to.

publish_to orders caller READS behind stage COMPUTE. Nothing ordered the other direction: buffers ALLOCATED on a stage stream (the verify’s returned logits/hidden, the VerifyCkpt stashes) are CONSUMED by kernels the caller enqueues on the PRIMARY stream, and when they drop, cudarc enqueues free_async on the ALLOCATING (stage) stream. With event tracking elided (the decode-path default) the drop carries no read-guard, so the pool can hand the block to the NEXT stage-stream allocation and its writes overwrite memory the queued primary-stream consumer has not read yet. Measured (research/pp2spec-crash-20260807): the spec round-seed read 13/4096 NaN = the uninitialized-bits signature (P(NaN|random u32) ~ 1/256), clean by host re-read time — a read-before-write race, fatal via the argmax-sentinel -> embed_gather MMU fault, and gated on c>=2 because a backed-up primary stream widens the window.

Fix law: before a ppN body enqueues NEW stage-stream work (allocations that may reuse freed blocks), every stage stream waits the caller’s stream at its current point. All primary consumers of the previous round’s stage-allocated buffers are enqueued by then (single host thread), so reuse-writes land strictly after them. Call at ppN-body ENTRY with the pre-enter caller stream. Door-shut configs never build a PpNRt, so single-card behavior is untouched.

pub fn record_done(&self) -> Result<CudaEvent, Box<dyn Error>>

Deferred readback: record a fresh completion event on the LAST stage’s stream (call after the step’s logits matmul has been enqueued there).

pub fn readback_stream(&self) -> &Arc<CudaStream>

The dedicated readback stream (last stage’s context).