pub struct Step35TokenGraphState {Show 24 fields
pub graphs: Vec<(usize, TokenGraph)>,
pub token_d: CudaSlice<u32>,
pub pos_d: CudaSlice<i32>,
pub logits_stage: CudaSlice<f32>,
pub x: CudaSlice<f32>,
pub x1: CudaSlice<f32>,
pub mixed_stage: CudaSlice<f32>,
pub sh_stage: CudaSlice<f32>,
pub k_shadow_stage: CudaSlice<f32>,
pub v_shadow_stage: CudaSlice<f32>,
pub router_logits: CudaSlice<f32>,
pub shexp_gate: CudaSlice<f32>,
pub shexp_up: CudaSlice<f32>,
pub shexp_act: CudaSlice<f32>,
pub gate_sig: CudaSlice<f32>,
pub dense_z: CudaSlice<f32>,
pub dense_gate: CudaSlice<f32>,
pub dense_up: CudaSlice<f32>,
pub dense_act: CudaSlice<f32>,
pub hn: CudaSlice<f32>,
pub probe_mixed: CudaSlice<f32>,
pub probe_x: CudaSlice<f32>,
pub token_hist: CudaSlice<u32>,
pub hist_idx: CudaSlice<i32>,
}Expand description
Whole-token decode graph state (step TP graph increment B). One stitched multi-device parent per fa bucket, plus the persistent host->graph plumbing: the device token id the in-graph embed gathers, the device position the ropes read (advanced in-graph), and the fixed logits stage the head writes.
Fields§
§graphs: Vec<(usize, TokenGraph)>(bucket_max, graph) — bucket keyed by the fa split geometry (fa_geom_eager).
token_d: CudaSlice<u32>§pos_d: CudaSlice<i32>§logits_stage: CudaSlice<f32>§x: CudaSlice<f32>Cross-child intermediates MUST live at fixed addresses (graph mem nodes remap at launch, so an alloc made inside one captured child is not referable from another): the running residual, the post-attention pair, the shared-expert row, and the e-context mirrors of the root-produced attention output and K/V shadow rows.
x1: CudaSlice<f32>§mixed_stage: CudaSlice<f32>§sh_stage: CudaSlice<f32>§k_shadow_stage: CudaSlice<f32>§v_shadow_stage: CudaSlice<f32>§router_logits: CudaSlice<f32>Alloc-free e-section scratch (child graphs cannot contain mem nodes): router logits, shared-expert gate/up/act rows + sigmoid scalar, dense-FFN z/gate/up/act, head hidden.
shexp_gate: CudaSlice<f32>§shexp_up: CudaSlice<f32>§shexp_act: CudaSlice<f32>§gate_sig: CudaSlice<f32>§dense_z: CudaSlice<f32>§dense_gate: CudaSlice<f32>§dense_up: CudaSlice<f32>§dense_act: CudaSlice<f32>§hn: CudaSlice<f32>§probe_mixed: CudaSlice<f32>MEMRA_TG_PROBE_LAYER diagnostics: capture-time copies of layer K’s attention output and post-FFN residual, dumped after replay for graph-vs-eager layer bisection.
probe_x: CudaSlice<f32>§token_hist: CudaSlice<u32>Chunk-loop (F-lite): device token history ring + its device write index, filled by the in-graph tail argmax chain; host reads the ring once per chunk.
hist_idx: CudaSlice<i32>