Skip to main content

Module spec_phase

Module spec_phase 

Source
Expand description

Per-burst spec-round phase attribution (MEMRA_SPEC_TRACE; glm5 alias honored) — the draft/verify/accept/rollback/maintenance split every spec family owns, with caller-tagged emit lines so banked receipts keep their grep shape. No CUDA deps beyond the stream drains at phase boundaries. Per-burst phase attribution for speculative rounds (MEMRA_SPEC_TRACE, generalized lane/glm5-extract-general from the glm5 loop’s MEMRA_GLM5_SPEC_TRACE — the alias stays honored). The draft / verify / accept / rollback / source-maintenance split is SPEC-FAMILY-GENERIC: any spec loop owns those five boundaries, and the level-2 verify sub-split buckets are MIXER-CLASS buckets (KDA, MLA — multi-family classes), not one model’s. The emit TAGS are the caller’s, so a family’s banked receipts keep their exact grep shape ([glm5-phase] / [glm5-phase-v] for the glm5 loop).

DEFAULT OFF BY DESIGN (the flag row’s law): each phase boundary SYNCHRONIZES the stream so device time lands in the right bucket, which serializes the round — a diagnostic instrument, never a serving mode, and its numbers are phase SHARES, not round walls (the un-traced round overlaps what the trace separates).

Structs§

SpecFirstTokenProf
The first-token phase buckets (ms) one served spec session carries until the worker prints its [spec-prof] line after the first burst. Every field is a wall interval bounded by stream drains on both sides, so the buckets are device-inclusive and additive; a bucket the route never runs stays 0.0.
SpecRoundProf
One verify round’s attribution row (MEMRA_SPEC_PROF=1, first SPEC_PROF_ROUNDS rounds of a session). Phase buckets are drained like the trace’s (shares under drains, so wall_ms is the round’s traced wall); k is the drafted count that entered the verify (after the confidence gate), j the accepted drafts, ctx the trunk rows at round entry, seq_rows the verify rows that took the PER-ROW mixer arm instead of the batched one (0 = every layer batched — a non-zero count at depth names the slow-path suspect by itself).
SpecRoundsLog
The per-session round log behind [spec-prof-rounds] / [spec-prof-summary].

Constants§

SPEC_PROF_ROUNDS
Rounds the per-round depth log keeps per session (lane/spec-route-depth-20260902).

Functions§

spec_prof_on
MEMRA_SPEC_PROF=1 (lane/b200-spec-ttft-20260902): the ONCE-PER-REQUEST first-token phase profile for a served spec session — every phase between the session’s prime and the first streamed token, in ms. Distinct from MEMRA_SPEC_TRACE (per-burst round SHARES): this instrument answers “where did the first-token latency go on THIS request”, so it buckets the one-time costs the round trace cannot see (cache alloc, target prime, boundary draw, drafter KV alloc, the round-1 drafter prime over the prompt) and the first burst’s wall. DEFAULT OFF BY DESIGN: phase boundaries synchronize the stream (the SpecPhaseNs::clock contract), so the traced first burst is a little slower than the untraced one; the line attributes, the untraced TTFT claims. Read once per process.
spec_trace_level
Trace LEVEL: 1 = the per-burst phase lines (draft/verify/accept/roll/maint); 2 = additionally the VERIFY sub-split — batched-class vs sequential-class time per burst (vkda with its in-kernel scan share, vmla, vrest = glue+FFN+head). Level 2 adds per-layer stream drains on top of level 1’s phase drains: shares, never walls, never a perf row (the standing trace law). Read once per process (the worker chunk-policy pattern). The general name wins when both names are set to DIFFERENT levels — with one loud stderr line naming the override (the alias is never silently dead).
spec_trace_on
MEMRA_SPEC_TRACE=1 (or the glm5 alias): per-burst phase attribution is on.