Struct SpecSession 

pub struct SpecSession {
    pub committed: Vec<u32>,
    pub next_pred: Option<u32>,
    pub sctr: u32,
    pub uctr: u32,
    pub pending_tok: Option<u32>,
    pub telem: SpecTelemetry,
    /* private fields */
}

Fields

committed: Vec<u32>

Every token whose state the caches hold, in order (prompt turns + generated), INCLUDING overshoot: spec commits accepted drafts past max_new; those rows are in the caches, so the session must count them. Callers render output from this, not from their own echo.

next_pred: Option<u32>

Greedy argmax predicting the token AFTER committed.last() (from the last turn’s final logits). Fuels empty-suffix continuation bursts (serve): the next turn emits this token first, feeds it, and the round loop resumes without any prime. None before the first turn.

sctr: u32

SAMPLED-SPEC stream continuity across bursts: Philox event counters persist here so a session’s randomness never repeats between generate_spec_session calls. (0,0) at admit.

uctr: u32

pending_tok: Option<u32>

PENDING-CARRY across bursts (2026-08-01, the serve burst-boundary fix): the bonus token emitted by the last round but NOT committed to the caches. The old tail committed it with a solo T=1 trunk pass (+ draft fill), and the next burst’s setup fed the stashed next_pred with ANOTHER solo pass — 2x ~11.5ms/burst measured on H100 q27 ([spec-setup] trace). Carrying it lets the next empty-suffix greedy burst consume it as round-0 verify col 0, exactly like a mid-burst full-accept boundary (no solo passes). INVARIANT: when set, committed (== cache rows) EXCLUDES this token although it was already emitted in the last burst’s output, and last_h holds the hidden of the last COMMITTED row (its predecessor — the chain-seed/fill anchor). next_pred is None (unknown without the commit pass). Non-empty-suffix or sampled turns must flush first (spec_flush_pending); generate_spec_session_sampled does this at entry, and serve parks only flushed sessions.

telem: SpecTelemetry

Session-lifetime acceptance telemetry (lane/accept-telemetry). Host-side u64 adds at the round accounting the loop already does — no syncs, no allocation. NOTE a pool-resumed session carries the PREVIOUS requests’ counts; per-request consumers diff with SpecTelemetry::delta_since around each burst.

Implementations

impl SpecSession

pub fn cache_max_ctx(&self) -> usize

Context capacity of the session’s caches (the server’s ContextFull guard).

pub fn rewind_pos(&self) -> Option<usize>

Committed position this session can REWIND to (its retained prompt-end boundary), if any. A request whose prompt matches committed[..pos] exactly can resume from here — see spec_rewind_to_checkpoint.

pub fn rewind_is_resident(&self) -> bool

Whether every ring-backed trunk/draft row needed by the retained checkpoint is resident.

pub fn demote_ready(&self) -> bool

Is this session in the DEMOTION-READY shape (see SpecSession::into_demoted)? false means a carried pending must be flushed first (spec_flush_pending), or the session has never run a turn and has no prediction to hand over.

pub fn has_pending(&self) -> bool

Does this session hold a carried pending bonus (flush required before a handoff/park)?

pub fn committed_len(&self) -> usize

Committed row count == cache rows (the session invariant), for the caller’s own fed-length cross-check at a handoff boundary.

pub fn into_demoted(self) -> Option<(Cache, u32)>

DEMOTION HANDOFF (lane/spec-gate, 2026-08-07): consume this session and hand its trunk cache + next-token prediction to the plain batched-decode path.

WHY THIS IS EXACT (greedy). The invariant at a burst boundary is cache.pos == committed.len(): every committed row has trunk KV + recurrent state, exactly as a plain tokenwise prime of the same committed sequence would have left it (that is the session-tail contract, and the same property spec_rewind_to_checkpoint and the reuse pool already rely on). next_pred is the argmax of the verify’s logits for the LAST committed row — and verify-column logits are bit-identical to plain decode’s logits at that position, because matmul_decode_exact bit-identity IS the basis of the greedy accept walk. So handing (cache, next_pred) to the batched path continues the stream from a state indistinguishable from one the batched path produced itself: the batched tick emits next_pred, feeds it into this same cache, and decodes on.

None when the session is not in the handoff shape — a carried pending (its bonus row is NOT in the cache, so spec_flush_pending must commit it first) or no next_pred yet (never bursted). Callers must not force it: a half-committed cache handed to the batched path would silently skip a token.

The MTP draft scratch, the persistent draft-graph context and the turn checkpoint are DROPPED here (freeing their VRAM): the batched path never drafts, and this handoff is one-way by design — there is no cheap symmetric re-promotion (rebuilding the draft KV would mean an mtp_kv_fill over the whole committed history).

pub fn reset_graph_fallback_on_resume(&mut self)

Pool-resume hook (audit Q2): clear the parked draft-graph failure memoization so a NEW request resuming this session gets one fresh capture chance — a transient-pressure capture failure must not persist for the pool’s whole lifetime (the TRT #16072 class). Logs once iff a flag was actually set; a no-fallback resume is silent and free.