Module spec 

Qwen3.5 MTP (NextN) greedy speculative decode (research/mtp/MTP-PLAN.md §A/§B/§C/§D).

Greedy spec decode is MATHEMATICALLY EXACT: the accepted+bonus token stream is token-for-token identical to plain greedy generate. This module provides:

• mtp_head_forward (§A, T=1): one NextN draft-token forward.
• decode_step_t (§D.3, T=K+1): batched target verify forward, all-column logits.
• generate_spec (§B): the draft/verify/accept/rollback orchestrator. Cache snapshot/rollback lives in cache.rs (§D.4). The MTP head uses its OWN scratch KV (§D.6), PERSISTENT over the committed sequence (see MtpScratch).

Structs

DsparkAnchorRecord

One compact, anchor-bounded DSpark supervision record. tokens[0] is the anchor at p and hidden is its predecessor carrier h[p-1], matching the live NextN/DSpark pairing. Target rows p..p+gamma-1 score tokens p+1..p+gamma. They are the full-target softmax’s top-k entries; target_tail_probs[j] is the probability mass outside those rows. All flattened target arrays are [gamma, top_k] in row-major order.

OptiForkGateStats

OptiForkStateIdentity

SpecBoundaryCapture

PREFIX-CACHE BOUNDARY CAPTURE (lane/spec-prefix-cache, 2026-08-14): the state a spec session records at its cold-prime split so the WORKER can publish a cross-request prefix entry — the commit-gated-publication port (research/cache-spec-design-20260814/PORT-PLAN.md item 1). Only the pieces that are DESTROYED by continuing the prime need copies here: the in-place GDN conv/ssm states (via Cache::snapshot, same mechanism as [SpecCheckpoint]) and the boundary logits. Full-attn KV rows [0..pos) and draft-scratch rows [0..pos) are append-only for the session’s lifetime (rollbacks never truncate below the prime boundary), so the worker slices those from the live caches post-burst instead of copying at prime time.

SpecSampling

Scratch KV for the MTP block (one full-attn layer).

SpecSession

SpecTelemetry

Always-on per-draft-position acceptance telemetry (lane/accept-telemetry, 2026-08-05 — the llama.cpp #26389 / vLLM spec-decode counter schema, upstream-sweeps 2026-08-05). Lives on the SpecSession and accumulates across bursts; the serve worker diffs a stashed copy per burst for its per-model /metrics aggregation and per-request usage. Same normalization as the [spec-stats] line: p-min-discarded chain tokens are counted in NEITHER drafted nor accepted.

Enums

OptiForkGateMode

Explicit OPTIPIPE diagnostic control. Forced modes are set only by optipipe-gate; the increment-2 controller can also be armed by the server’s fresh-process research door.

Constants

PEN_WINDOW_MAX

CEILING on the penalty window, and it is a COST bound, not a semantic preference. penalize_logits_f32 (cu/spec_sample.cu) dedups on device by having thread i scan hist[0..i], so a pass is O(n_hist²) and it runs ~3x per verify round (the q rows, the p column, the bonus column). The serve API arms penalty_last_n = usize::MAX for ANY non-identity penalty — “the whole context”, llama’s repeat_last_n = -1 — so an uncapped session window would put a 128k-token history through that kernel: ~1.7e10 comparisons per pass, tens of ms per round, i.e. penalties would silently destroy decode throughput on exactly the long-context requests that most want them. 8192 keeps a pass at ~7e7 comparisons (tens of microseconds) while still being 128x wider than the pre-lane effective window (64 prompt-tail tokens + whatever the current burst had generated). A request that genuinely needs a window beyond this wants host-side dedup + counts through a new kernel signature — a follow-up lane, named here rather than hidden. pub since lane/dspark-penalized-sampled-20260821: the dspark route’s accept walk and the dspark_sample_gate binary trim their uploads with the SAME cap — a second constant is a second thing to drift.

SPEC_TELEM_POS

Tracked draft positions for SpecTelemetry (serve K defaults to 3; the run-spec gate sweeps K=1..8, and MEMRA_SPEC_CAPMAX defaults to 7 — 8 covers every tuned config).

Traits

SpecConstraint

GRAMMAR HOOK for constrained spec decode (lane/constrained-full, 2026-08-03). The engine stays llguidance-agnostic: the server adapts its per-session grammar state behind this trait. CONTRACT (the verify-side truncation rule — token-identical to constrained plain greedy decode): the exactness walk runs UNMASKED first; the hook then (a) truncates acceptance at the first grammar-illegal accepted token, and (b) when the truncation fired or the bonus is illegal, the engine recomputes that slot as the MASKED argmax of the target’s own verify column (an unmasked argmax that is grammar-legal IS the masked argmax — masking only removes tokens — so the common case pays nothing). consume advances the state with each EMITTED token in order; EOS handling is the implementor’s job (skip).

Functions

optipipe_gate_stats

pen_window_seed

Seed a penalty window over the SESSION, not the burst (lane/sampled-spec-quality, Item 2). The window is the last max(penalty_last_n, 64) tokens of session_committed ++ burst_prompt — for a cold turn-1 burst (session_committed empty, default penalty_last_n) that is byte-identically the pre-lane prompt.iter().rev().take(64).rev(); for a continuation burst it is the stream the client actually asked us to penalize, where the pre-lane code had NOTHING. pub since lane/dspark-penalized-sampled-20260821: the dspark route seeds its session window through the SAME function (one definition of “the window” across both spec routes and the gate binary’s trunk-only reference arm).

reset_optipipe_gate_stats

sample_boundary_token

Host-row twin of sample_boundary_token_dev (the prime / feed / entry rows arrive as host Vec<f32>).

sample_boundary_token_dev

Draw a BOUNDARY token from the target distribution the request asked for (lane/sampled-spec-quality, Item 1) — the fix for “sampled spec emits an ARGMAX token at every burst boundary”.

set_optipipe_controller_threshold

Arm the increment-2 diagnostic controller. The threshold applies to the uncalibrated two-token draft-probability product. Serving can call this only through its explicit fresh-process research door; the absent-door default remains byte-for-byte disabled.

set_optipipe_gate_mode

Arm or disarm the forced harness. Serving uses only set_optipipe_controller_threshold.

spec_pen_session_on

ROLLBACK DOOR for SESSION-SPANNING penalty history (lane/sampled-spec-quality). Default ON: pen_hist is seeded from the session’s committed tail, so repetition / frequency / presence penalties see the whole stream. MEMRA_SPEC_PEN_SESSION=0 restores the pre-lane posture (each burst restarts the window from its own prompt slice, i.e. from NOTHING on a continuation burst) — and with the door shut the worker must keep refusing penalized sampled prefix-cache restores, because the restored session’s continuation burst is handed no prompt slice at all.

spec_replay_env_enabled

spec_replay_env_on

Parse the documented MEMRA_SPEC_REPLAY=1 rollback seam.

spec_restore_republish_on

ROLLBACK DOOR for extended-entry publication from a RESTORED session (lane/sampled-spec-quality, Item 3). Default ON: a converted prefix-cache hit that fed a suffix captures its own prompt-end boundary so the NEXT turn can hit a longer prefix. MEMRA_SPEC_RESTORE_REPUBLISH=0 restores the pre-lane posture (a namespace learns exactly one boundary and never advances it). Whole-entry semantics only — the boundary is the restored session’s own prompt end, so entry_pos != fed_len still refuses on the way in.

spec_sampled_boundary_on

ROLLBACK DOOR for sampled BOUNDARY tokens (lane/sampled-spec-quality, 2026-08-19). Default ON: the token a burst emits at its own boundary is drawn from the request’s sampler. MEMRA_SPEC_SAMPLED_BOUNDARY=0 restores the pre-lane posture (an ARGMAX at every boundary) without touching greedy, which is byte-unaffected either way.