Skip to main content

GemmaSpecSession

Struct GemmaSpecSession 

Source
pub struct GemmaSpecSession {
    pub cache: Cache,
    pub committed: Vec<u32>,
    pub prompt_len: usize,
    pub rounds: usize,
    pub drafted: usize,
    pub accepted: usize,
    /* private fields */
}
Expand description

BURST-SCOPED gemma4 spec session (lane/gemma-batched stage 1, 2026-08-16): the serve twin of generate_spec_gemma. That function is GENERATION-scoped — it builds its own cache, primes, loops to completion, and its break 'outer exits deliberately skip the final round’s rollback/h/pending updates (safe only because the cache dies with the call). A served session must instead stop and RESUME across scheduler ticks, so this type carries the exact cross-round state the eager loop threads through its locals:

  • cache — the trunk Cache; rows = committed (prompt + emitted, INCL. overshoot).
  • h — post-output_norm hidden of the LAST committed row (device; draft seed).
  • pending— the last local: the predicted next token. Emitted as the FIRST token of the next round and appended as verify col 0 there; it has NO cache row while parked here (the Q38 next_pred convention).
  • kc_next/prev_full — the adaptive-depth + self-keyed in-round-cut carries.

BOUNDARY LAW (the Q38 pending-carry/empty-suffix bug class, banked as gate cases in gemma-spec-session-gate before this was written): a burst NEVER exits mid-round. Every round runs to completion — emission, rollback to the accepted prefix, h/pending update, trim-adapt learn — and only then does the burst-target check run. Overshoot past target is committed and returned (the caller clamps VISIBLE emission; state counts every row, exactly like Q38’s SpecSession::committed). EOS ends the burst at its round boundary with the same complete-state guarantee.

V1 scope (greedy serve): EAGER round arm only — the round-graph / burst-ring arms are generation-scoped perf doors (their ring/pos-counter state does not checkpoint at round boundaries) and the shipping bench receipts (154.9/176-179, ASSISTANT-ARM- RESULTS.md) were measured on this same eager arm. Dense gemma4 only (E4B refused). Fresh session per request: no prefix reuse, no multi-turn suffix — continuation bursts are always empty-suffix by construction.

Fields§

§cache: Cache§committed: Vec<u32>

Every token whose rows the cache holds, in order (prompt + emitted, incl. overshoot).

§prompt_len: usize§rounds: usize

Session-lifetime spec telemetry (rounds / drafted / accepted).

§drafted: usize§accepted: usize

Implementations§

Source§

impl GemmaSpecSession

Source

pub fn emitted_len(&self) -> usize

Tokens the session has emitted (committed past the prompt). The pending token is NOT included — it has no cache row and the next burst emits it first.

Source

pub fn cache_max_ctx(&self) -> usize

Context capacity of the session’s cache (the server’s ContextFull guard).

Source

pub fn into_demoted(self) -> (Cache, u32, Vec<u32>)

DEMOTE HANDOFF (stage-2 seam, gated by the session gate’s demote case): hand the trunk cache to the plain path. The cache rows are exactly committed (boundary law), and the pending token is returned as the plain path’s device_next-equivalent — the plain loop feeds it as its first decode input. The draft side holds no per-session state (the assistant drafter reads the TRUNK’s KV; trim-adapt is model-lifetime, not session), so dropping self is the whole handoff.

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.