Struct SpecSession 

pub struct SpecSession {
    pub committed: Vec<u32>,
    pub next_pred: Option<u32>,
    pub sctr: u32,
    pub uctr: u32,
    pub pending_tok: Option<u32>,
    pub capture_at: Option<usize>,
    pub boundary_captures: Vec<SpecBoundaryCapture>,
    pub ckpt_at: Option<usize>,
    pub capture_disabled: bool,
    /* private fields */
}

Fields

committed: Vec<u32>

Every token whose state the caches hold, in order (prompt turns + generated), INCLUDING overshoot: spec commits accepted drafts past max_new; those rows are in the caches, so the session must count them. Callers render output from this, not from their own echo.

next_pred: Option<u32>

Greedy argmax predicting the token AFTER committed.last() (from the last turn’s final logits). Fuels empty-suffix continuation bursts (serve): the next turn emits this token first, feeds it, and the round loop resumes without any prime. None before the first turn.

sctr: u32

SAMPLED-SPEC stream continuity across bursts: Philox event counters persist here so a session’s randomness never repeats between generate_spec_session calls. (0,0) at admit.

uctr: u32

pending_tok: Option<u32>

PENDING-CARRY across bursts (2026-08-01, the serve burst-boundary fix): the bonus token emitted by the last round but NOT committed to the caches. The old tail committed it with a solo T=1 trunk pass (+ draft fill), and the next burst’s setup fed the stashed next_pred with ANOTHER solo pass — 2x ~11.5ms/burst measured on H100 q27 ([spec-setup] trace). Carrying it lets the next empty-suffix greedy burst consume it as round-0 verify col 0, exactly like a mid-burst full-accept boundary (no solo passes). INVARIANT: when set, committed (== cache rows) EXCLUDES this token although it was already emitted in the last burst’s output, and last_h holds the hidden of the last COMMITTED row (its predecessor — the chain-seed/fill anchor). next_pred is None (unknown without the commit pass). Non-empty-suffix or sampled turns must flush first (spec_flush_pending); generate_spec_session_sampled does this at entry, and serve parks only flushed sessions.

capture_at: Option<usize>

PREFIX-CACHE publication request (lane/spec-prefix-cache): worker sets this to the miss-LCP boundary before a cold burst; the prime captures at exactly that split (it must coincide with the burst’s prime_split or no capture happens). One-shot: consumed by the prime, result lands in boundary_captures.

boundary_captures: Vec<SpecBoundaryCapture>

The captures the last prime produced (see SpecBoundaryCapture). Worker drains them post-burst to assemble prefix entries. A failed capture is silent, like turn_ckpt — publication just isn’t available for that request. Plural since lane/frspec-multiturn-cache (2026-08-21): a cold burst can capture BOTH the miss-LCP split (the shared-prefix class) and the stable pre-generation boundary (the next-turn re-render class) — one entry per stop, exactly the boundary set the plain prefill tick publishes/checkpoints.

ckpt_at: Option<usize>

STABLE-BOUNDARY TURN CHECKPOINT REQUEST (lane/frspec-multiturn-cache, 2026-08-21): the ABSOLUTE committed-length position the next non-empty prime should capture turn_ckpt at, instead of prompt-end. The worker sets it to the STABLE PRE-GENERATION boundary (plain_checkpoint_boundary — before the live generation header the client rewrites), porting the 2026-08-09 plain-tier fix: a prompt-end spec checkpoint includes the template’s live assistant-generation header (<|im_start|>assistant\n<think>\n), which the NEXT turn’s re-render replaces, so affinity_match diverged a couple tokens below the checkpoint and the spec pool declined 100% of multi-turn agent traffic (measured: spec-affinity: declined (history diverged at 6811 of checkpoint 6813), research/multiturn-cache-20260821 B4). One-shot, capture_at convention; None = legacy prompt-end capture.

capture_disabled: bool

FAIL-SAFE (lane/step37-vram-admission-20260830, external-review corroboration): set by the worker on a session serving a step-OOM park REPLAY. The burst entry pre-marks the draft-graph fallback so the replay never re-enters the capture path — the capture appetite is part of what drove the card to the OOM, and a replay that recaptures re-runs the incident. If the eager replay still cannot fit, the bounded retry budget exhausts into the honest recoverable Overloaded error instead of looping.

Implementations

impl SpecSession

pub fn cache_max_ctx(&self) -> usize

Context capacity of the session’s caches (the server’s ContextFull guard).

pub fn cache_ref(&self) -> &Cache

Read access to the live trunk cache (lane/spec-prefix-cache): the worker slices full-attn KV rows [0..capture.pos) out of it when publishing a boundary capture — those rows are append-only for the session’s lifetime (rollbacks never truncate below the prime boundary), so no copy was taken at prime time.

pub fn draft_plane_ref( &self, ) -> Option<(&CudaSlice<u8>, &CudaSlice<u8>, usize, usize)>

Read access to the persistent draft-scratch plane (lane/spec-on-cache-hit): the worker slices rows [0..capture.pos) when publishing a boundary capture, exactly like the trunk KV — draft rows below the prompt end are append-only for the session’s lifetime (the prime fill wrote them once; rollbacks reset len_d to the committed length, never below the prime boundary, and the true-hidden refresh rewrites generated positions only). Returns (k, v, k_tok_bytes, v_tok_bytes). None when the scratch is ring-backed (Step35 SWA — physical rows are not prefix-addressable; the prefix cache already refuses that class end to end).

pub fn telemetry(&self) -> SpecTelemetry

Snapshot the session’s process-local acceptance counters for per-burst diffing.

pub fn rewind_pos(&self) -> Option<usize>

Committed position this session can REWIND to (its retained prompt-end boundary), if any. A request whose prompt matches committed[..pos] exactly can resume from here — see spec_rewind_to_checkpoint.

pub fn rewind_is_resident(&self) -> bool

Whether every ring-backed trunk/draft row needed by the retained checkpoint is resident.

pub fn demote_ready(&self) -> bool

Is this session in the DEMOTION-READY shape (see SpecSession::into_demoted)? false means a carried pending must be flushed first (spec_flush_pending), or the session has never run a turn and has no prediction to hand over.

pub fn has_pending(&self) -> bool

Does this session hold a carried pending bonus (flush required before a handoff/park)?

pub fn committed_len(&self) -> usize

Committed row count == cache rows (the session invariant), for the caller’s own fed-length cross-check at a handoff boundary.

pub fn into_demoted(self) -> Option<(Cache, u32)>

DEMOTION HANDOFF (lane/spec-gate, 2026-08-07): consume this session and hand its trunk cache + next-token prediction to the plain batched-decode path.

WHY THIS IS EXACT (greedy). The invariant at a burst boundary is cache.pos == committed.len(): every committed row has trunk KV + recurrent state, exactly as a plain tokenwise prime of the same committed sequence would have left it (that is the session-tail contract, and the same property spec_rewind_to_checkpoint and the reuse pool already rely on). next_pred is the argmax of the verify’s logits for the LAST committed row — and verify-column logits are bit-identical to plain decode’s logits at that position, because matmul_decode_exact bit-identity IS the basis of the greedy accept walk. So handing (cache, next_pred) to the batched path continues the stream from a state indistinguishable from one the batched path produced itself: the batched tick emits next_pred, feeds it into this same cache, and decodes on.

None when the session is not in the handoff shape — a carried pending (its bonus row is NOT in the cache, so spec_flush_pending must commit it first) or no next_pred yet (never bursted). Callers must not force it: a half-committed cache handed to the batched path would silently skip a token.

The MTP draft scratch, the persistent draft-graph context and the turn checkpoint are DROPPED here (freeing their VRAM): the batched path never drafts, and this handoff is one-way by design — there is no cheap symmetric re-promotion (rebuilding the draft KV would mean an mtp_kv_fill over the whole committed history).

pub fn reset_graph_fallback_on_resume(&mut self)

Pool-resume hook (audit Q2): clear the parked draft-graph failure memoization so a NEW request resuming this session gets one fresh capture chance — a transient-pressure capture failure must not persist for the pool’s whole lifetime (the TRT #16072 class). Logs once iff a flag was actually set; a no-fallback resume is silent and free.