pub struct GemmaSpecSession {
pub cache: Cache,
pub committed: Vec<u32>,
pub prompt_len: usize,
pub rounds: usize,
pub drafted: usize,
pub accepted: usize,
/* private fields */
}Expand description
BURST-SCOPED gemma4 spec session (lane/gemma-batched stage 1, 2026-08-16): the serve
twin of generate_spec_gemma. That function is GENERATION-scoped — it builds its own
cache, primes, loops to completion, and its break 'outer exits deliberately skip the
final round’s rollback/h/pending updates (safe only because the cache dies with the
call). A served session must instead stop and RESUME across scheduler ticks, so this
type carries the exact cross-round state the eager loop threads through its locals:
cache— the trunk Cache; rows =committed(prompt + emitted, INCL. overshoot).h— post-output_norm hidden of the LAST committed row (device; draft seed).pending— thelastlocal: the predicted next token. Emitted as the FIRST token of the next round and appended as verify col 0 there; it has NO cache row while parked here (the Q38next_predconvention).kc_next/prev_full— the adaptive-depth + self-keyed in-round-cut carries.
BOUNDARY LAW (the Q38 pending-carry/empty-suffix bug class, banked as gate cases in
gemma-spec-session-gate before this was written): a burst NEVER exits mid-round.
Every round runs to completion — emission, rollback to the accepted prefix, h/pending
update, trim-adapt learn — and only then does the burst-target check run. Overshoot
past target is committed and returned (the caller clamps VISIBLE emission; state
counts every row, exactly like Q38’s SpecSession::committed). EOS ends the burst at
its round boundary with the same complete-state guarantee.
V1 scope (greedy serve): EAGER round arm only — the round-graph / burst-ring arms are generation-scoped perf doors (their ring/pos-counter state does not checkpoint at round boundaries) and the shipping bench receipts (154.9/176-179, ASSISTANT-ARM- RESULTS.md) were measured on this same eager arm. Dense gemma4 only (E4B refused). Fresh session per request: no prefix reuse, no multi-turn suffix — continuation bursts are always empty-suffix by construction.
Fields§
§cache: Cache§committed: Vec<u32>Every token whose rows the cache holds, in order (prompt + emitted, incl. overshoot).
prompt_len: usize§rounds: usizeSession-lifetime spec telemetry (rounds / drafted / accepted).
drafted: usize§accepted: usizeImplementations§
Source§impl GemmaSpecSession
impl GemmaSpecSession
Sourcepub fn emitted_len(&self) -> usize
pub fn emitted_len(&self) -> usize
Tokens the session has emitted (committed past the prompt). The pending token is NOT included — it has no cache row and the next burst emits it first.
Sourcepub fn cache_max_ctx(&self) -> usize
pub fn cache_max_ctx(&self) -> usize
Context capacity of the session’s cache (the server’s ContextFull guard).
Sourcepub fn into_demoted(self) -> (Cache, u32, Vec<u32>)
pub fn into_demoted(self) -> (Cache, u32, Vec<u32>)
DEMOTE HANDOFF (stage-2 seam, gated by the session gate’s demote case): hand the
trunk cache to the plain path. The cache rows are exactly committed (boundary
law), and the pending token is returned as the plain path’s device_next-equivalent
— the plain loop feeds it as its first decode input. The draft side holds no
per-session state (the assistant drafter reads the TRUNK’s KV; trim-adapt is
model-lifetime, not session), so dropping self is the whole handoff.