Module decode 

Incremental decode (T=1) with the dual cache + greedy generation loop. Serves end-to-end. Reuses the validated kernels; threads KV (full-attn) and conv/SSM state (linear-attn) across steps.

Structs

GenOutput

Result of generate_with: the generated token ids + why it stopped.

GenParams

Generation parameters for the reusable serving API (generate_with).

GraphDecodeState

Persistent CUDA-graph decode state (CUDA-GRAPH-PLAN Phase 3). Holds the device-resident counters the captured graph reads/writes (token_d = current/next token id, pos_d = rope position) — both at FIXED addresses baked into every captured graph — plus the per-t_kv-bucket graph cache. The bucket key is the eager (fa_vec, n_splits) pair (see Engine::fa_bucket_key): every t_kv that maps to the same key reproduces eager’s split geometry, so one captured graph replays bit-identically for the whole bucket. A new key triggers a re-capture (n_splits changes ~every 64 tokens).

GraphSession

Long-lived step-wise CUDA-graph decode session (see HybridModel::graph_session_new). One replay per step(); the only steady-state D2H is the 4-byte next-token read.

Hy3Layer0Stages

Diagnostic-only snapshots of Hy3 layer 0 in the eager T=1 serving path. Each buffer is one residual-width device row captured before the next stage can reuse it.

Enums

StopReason

Why generation stopped.