Struct Dsv4Gpu 

pub struct Dsv4Gpu {

    pub model: Dsv4Model,
    pub stages: Vec<Stage>,
    pub layer_stage: Vec<usize>,
    pub split_at: u32,
    pub max_seq: usize,
    pub variant: ActQuantVariant,
    pub fc_yarn_host: FreqsCis,
    pub fc_plain_host: FreqsCis,
    pub mtp: Option<MtpDev>,
    pub dspark: Option<DsparkDev>,
    pub expert_arm: ExpertArm,
    pub decode_path: DecodePath,
    pub dots_f32: bool,
    pub chains_f32: bool,
    pub dspark_head_f32: bool,
    pub dense_fp8: bool,
    /* private fields */
}

Fields

model: Dsv4Model

stages: Vec<Stage>

layer_stage: Vec<usize>

split_at: u32

max_seq: usize

variant: ActQuantVariant

fc_yarn_host: FreqsCis

fc_plain_host: FreqsCis

mtp: Option<MtpDev>

dspark: Option<DsparkDev>

iteration 3: the DSpark drafter (0731 lineage), loaded under MEMRA_DSV4_DRAFTER=dspark; None = today’s exact behavior everywhere.

expert_arm: ExpertArm

decode_path: DecodePath

dots_f32: bool

lane 9 (owner ruling 2026-08-19): island dots on the DEVICE decode path run the f32-accumulation serving arm when true (fork-gated); false = the f64 oracle-truth arm (MEMRA_DSV4_DOTS_ARM=f64). Legacy path and prefill NEVER consult this (they stay the pinned reference realizations).

chains_f32: bool

0731 re-gate extension rung — RATIFIED by the owner 2026-08-19 and now the DEFAULT (unset env == f32x): the remaining f64 dependency chains on the DEVICE decode path (sink scores/soft/out, rmsnorm, headrms, rowsq_scale, indexer_score) run f32-accumulation twins when true. false = those chains keep the f64 kernels (MEMRA_DSV4_DOTS_ARM=f64|f32 — oracle/debug arms, bytes untouched). hc_sinkhorn is NOT in f32x (never authorized). Legacy path and prefill NEVER consult this.

dspark_head_f32: bool

iteration-3 rung 4c MEASURED FORK (MEMRA_DSV4_DSPARK_HEAD_ARM=f32x, default f64 = the lane-10-gated bytes): the DSpark drafter’s shared-trunk-head projection over block_size rows uses the f32-accumulation hoisted kernel instead of the f64 one. Affects WHICH tokens are drafted, never the emitted stream (verification always emits the trunk’s own argmax — the greedy identity law).

dense_fp8: bool

iteration-5 FP8 dense arm (MEMRA_DSV4_DENSE_ARM; DEFAULT fp8 on the device decode path since the 2026-08-20 ratification, bf16 selectable and the legacy default): the DEVICE decode/verify paths read the FP8-blk linears as-stored (e4m3 + f32 block scales) through the bit-identical GEMV twins, halving the dense weight traffic (79.9% of a step’s bytes). It5 ledger item 3: the trunk bf16 slabs are NOT device-resident under this arm — they hold DenseBf16::Host staged residency (same bytes, staged H2D per prefill pass); the legacy path is a boot refusal and the drafter’s cuBLASLt linears keep resident bf16 (no twins).

Implementations

impl Dsv4Gpu

pub fn load( dir: &Path, devices: &[usize], variant: ActQuantVariant, max_seq: usize, ) -> Result<Self, String>

Open the artifact and place the trunk across devices. split_at = first layer of stage 1, derived from per-layer byte math unless overridden.

pub fn vram_report(&self) -> Result<Vec<(usize, u64, u64, u64)>, String>

(free, total, resident-by-loader) bytes per device — the placement table source.

pub fn forward( &self, ids: &[u32], capture: Option<&mut GpuCapture>, early_exit_after: Option<u32>, ) -> Result<Option<ForwardOut>, String>

Full trunk prefill. Returns last-position logits, or None on early exit. early_exit_after stops after that layer (fixture Input B replays layers 0..=3).

pub fn prefill_with_cache( &self, ids: &[u32], state: &mut DecodeState, ) -> Result<ForwardOut, String>

Lane 6: prefill the prompt with the lane-4 path while POPULATING the decode caches, so decode_step can continue incrementally from ids.len().

pub fn mtp_logits_last( &self, h_trunk: &CudaSlice<f32>, ids: &[u32], ) -> Result<Vec<f32>, String>

MTP logits at the fixture call shape (model.py:826 — same ids to trunk and MTP; the V3 NextN drafter shift is the spec-decode lane’s wiring, not claimed here). h_trunk = the trunk’s final hc state on the LAST stage (ForwardOut::h_last).

pub fn mtp_logits_last_cap( &self, h_trunk: &CudaSlice<f32>, ids: &[u32], capture: Option<&mut GpuCapture>, ) -> Result<Vec<f32>, String>

Self::mtp_logits_last with a capture pass-through (lane 7: the native-GEMM kernel gate captures the MTP block’s moe_x under want = {n_trunk}).

pub fn trunk_logits_row( &self, h: &CudaSlice<f32>, s: usize, row: usize, ) -> Result<Vec<f32>, String>

Trunk-head logits at position row of a ForwardOut hc state (m-sensitivity probe).

pub fn alloc_decode_state(&self) -> Result<DecodeState, String>

Allocate the per-layer decode caches (capacity = max_seq, the reference register_buffer shape) on each layer’s owning stage. Returns a fresh state (pos = 0) ready for Self::prefill_with_cache.

pub fn decode_step( &self, tok: u32, state: &mut DecodeState, ) -> Result<Vec<f32>, String>

One incremental decode step: consume tok at position state.pos through all trunk layers + head using the caches (hc state carried across the PP boundary by host bounce, one copy per step). Returns the full logits row predicting position state.pos + 1.

pub fn decode_step_probe( &self, tok: u32, state: &mut DecodeState, ) -> Result<(Vec<f32>, Vec<(String, Vec<f32>)>), String>

Diagnostic twin: returns (logits, named per-layer intermediates).

pub fn decode_step_greedy( &self, tok: u32, state: &mut DecodeState, ) -> Result<u32, String>

Greedy decode step (bench serving shape): returns ONLY the next token; on the device path the argmax runs on-device and 4 bytes cross back. Legacy path falls back to the full-logits step + host argmax (same value by the argmax tie-rule equivalence).

impl Dsv4Gpu

pub fn dspark_alloc_state(&self) -> Result<DsparkState, String>

Allocate the drafter decode state on the last stage: 3 rings [win + block, hd] (ring + transient draft rows, struct doc) + the tap rows [block+1, n_t*hidden].

pub fn dspark_prime_prefill( &self, state: &mut DsparkState, main_hidden: &CudaSlice<f32>, s: usize, ) -> Result<(), String>

Prefill ring priming (M:763-769): last min(s, win) positions land at slot p % win. main_hidden = [s, n_t*hidden] tap rows from the prefill.

pub fn dspark_prefill_prime( &self, ids: &[u32], state: &mut DecodeState, dstate: &mut DsparkState, ) -> Result<ForwardOut, String>

Trunk prefill + DSpark ring prime in ONE pass — the device twin of the CPU oracle’s trunk.forward(&seq[..p0], 0) + dspark.prime_prefill(&pre.main_hidden, p0) pair (dsv4_dspark_gate components mode).

The prefill taps come from the existing GpuCapture::layer_out hook (the target layers’ full hc state [s, hc, hidden]), then run through the SAME memra_dsv4_hc_mean kernel the decode tap uses — prefill and decode taps must not be two numeric realizations of one tap — and are placed at the target’s concat stride with place_cols, reproducing the oracle’s main_hidden[(p*n_t + k)*hidden ..] layout exactly.

pub fn dspark_write_rings( &self, state: &mut DsparkState, tap_row: usize, pos: usize, ) -> Result<(), String>

Ring advance for ONE committed position (§3.1 drafter rule: accepted positions only). tap_row indexes into state.taps (the row that holds position pos’s hc-mean concat).

pub fn dspark_forward_spec( &self, state: &mut DsparkState, input_token: u32, tap_row: usize, pos: usize, capture: bool, ) -> Result<DsparkProposal, String>

forward_spec (M:928-936) + forward_head (M:860-874) on the device: ONE parallel noise-block draft through the 3 blocks, shared trunk head over all block rows, sequential rank-256 markov chaining (greedy), fp32 confidence. Mutates ONLY the rings’ transient rows (drafting is side-effect-free on trunk + persistent ring state — §3.1). tap_row = the taps row holding position pos’s hc-mean concat.

pub fn decode_step_tap( &self, tok: u32, state: &mut DecodeState, dspark_state: &mut DsparkState, tap_row: usize, ) -> Result<Vec<f32>, String>

Device decode step + the DSpark tap into dspark_state.taps row tap_row (full logits — the gates’ contract).

pub fn decode_step_greedy_tap( &self, tok: u32, state: &mut DecodeState, dspark_state: &mut DsparkState, tap_row: usize, ) -> Result<u32, String>

Greedy twin of Self::decode_step_tap (device argmax, 4-byte D2H).

impl Dsv4Gpu

pub fn verify_tmax(&self) -> usize

Verify-round depth ceiling: block_size + 1 with the drafter loaded, else 0 (and then no transient rows are reserved anywhere — today’s exact allocation).

pub fn alloc_verify_state(&self) -> Result<VerifyState, String>

Allocate the batched-verify state (arenas + §3.1 checkpoints). Requires the drafter (the only producer of rounds) and the device decode path.

impl Dsv4Gpu

pub fn verify_batch_dev( &self, toks: &[u32], state: &mut DecodeState, vstate: &mut VerifyState, taps: Option<&mut CudaSlice<f32>>, want_logits: bool, ) -> Result<(Option<Vec<f32>>, Vec<u32>), String>

Batched T=k+1 verify forward (§3.1): ONE trunk pass over toks at positions state.pos .. state.pos+T-1, logits for EVERY position (the accept walk needs them all), state advanced PROVISIONALLY for all T. Exactly one Self::commit_verify_dev must follow, which makes the accepted prefix permanent and rolls the rest back. The DSpark trunk tap is written for all T rows when taps is Some (rows 0..T-1 of the drafter’s taps buffer).

Returns (logits [T, vocab] when want_logits, per-position argmax [T]).

pub fn commit_verify_dev( &self, state: &mut DecodeState, vstate: &mut VerifyState, n_commit: usize, ) -> Result<(), String>

Commit the first n_commit positions of the open round and roll the rest back (§3.1 invariant: every trunk cache class ends bit-identical to plain sequential decode of exactly the committed positions). Ring slots take their transient rows; the compressors replay; the append-only stores fall back to their high-water mark.

pub fn spec_greedy_batched_with( &self, prompt: &[u32], n_new: usize, state: &mut DecodeState, dstate: &mut DsparkState, vstate: &mut VerifyState, ) -> Result<SpecRunGpu, String>

The device propose-then-verify greedy loop with BATCHED verification — the engine-side twin of spec_oracle::run_spec_greedy_batched, including its round/budget accounting (the budget-truncated final round and its pending-carry no-propose tail), so proposal streams and token streams are comparable item-for-item with the CPU oracle’s.

Greedy law: the trunk’s own argmax is ALWAYS the emitted token, so the output stream is plain greedy by construction — and because every batched kernel on this path is bit-exact against its single-position twin, that identity is byte-exact on device too, not merely mathematical. Reads the MEMRA_DSV4_SPEC_DEPTH knob and delegates to Self::spec_greedy_batched_depth. Every existing gate and bench calls this form, so their behaviour is decided by the environment exactly as before.

pub fn spec_greedy_batched_depth( &self, prompt: &[u32], n_new: usize, state: &mut DecodeState, dstate: &mut DsparkState, vstate: &mut VerifyState, depth_cap: usize, ) -> Result<SpecRunGpu, String>

Self::spec_greedy_batched_with with the verify-depth ceiling passed explicitly. usize::MAX means “no cap” (the drafter’s own block_size + 1).

Greedy identity is preserved at every cap by construction: truncating the proposal only shortens the accepted prefix, and the head token of every round is the trunk’s own argmax. That is what makes a depth sweep measurable without re-earning the identity law at each rung – though the sweep still asserts it per arm.

pub fn spec_greedy_batched( &self, prompt: &[u32], n_new: usize, ) -> Result<SpecRunGpu, String>

Self::spec_greedy_batched_with with freshly allocated state (gate shape).

impl Dsv4Gpu

pub fn cache_classes( &self, state: &DecodeState, ) -> Result<Vec<(String, Vec<f32>)>, String>

Every LIVE trunk cache class, per layer, as host f32 arrays — the instrument for the §3.1 device state gate (batched round + commit vs plain sequential decode of the committed tokens, bit for bit). “Live” is load-bearing: bytes past n_blocks in an append-only store, and the TRANSIENT verify rows, are dead scratch and are deliberately excluded (the CPU-oracle gate draws the same line).

pub fn dspark_ring_classes( &self, dstate: &DsparkState, ) -> Result<Vec<(String, Vec<f32>)>, String>

The DSpark drafter’s main_kv rings as host f32 arrays (accepted-position-only ring-write rule gate: the batched drafted arm’s rings must end bit-identical to a plain greedy run that wrote a ring row at EVERY decoded position).