Struct DecodeFlashSpec 

pub struct DecodeFlashSpec {
    pub q: NodeId,
    pub k: NodeId,
    pub v: NodeId,
    pub alibi: Option<NodeId>,
    pub softmax_scale: f32,
    pub causal: bool,
    pub window_size_left: Option<usize>,
    pub window_size_right: Option<usize>,
    pub softcap: Option<f32>,
    pub k_len: DynScalar,
    pub decomposed_out: NodeId,
    pub reconverge: NodeId,
}

The decode-attention region the model hands to the optimizer as a flash-arm candidate. Arm 0 — the decomposed region — is already in the graph; this describes how to build arm 1 (the flash kernel) and where to splice the Op::Branch.

q is both the flash node’s first input and the branch’s shared diverge point (the decomposed chain and the flash node both depart from q). decomposed_out is arm 0’s exit (the region’s materialized output, [B, Hq, Sq, D] — same shape/dtype as the flash output). reconverge is the sole consumer of decomposed_out (the merge).

Fields

q: NodeId

q — [B, Hq, Sq, D]. Also the branch diverge point.

k: NodeId

k — the capacity KV buffer [B, Hkv, capacity, D].

v: NodeId

v — the capacity KV buffer [B, Hkv, capacity, D].

alibi: Option<NodeId>

Optional ALiBi slopes [Hq]. Its presence disqualifies the arm (the capacity-K CUDA kernel has no ALiBi param) — carried so the gate can reject it explicitly.

softmax_scale: f32

1/sqrt(head_dim) (or the model’s scale).

causal: bool

Causal masking. Accepted-and-ignored by the kernel (seq_q == 1 attends the full [0, k_len) prefix), so it does not disqualify.

window_size_left: Option<usize>

Left sliding-window bound — disqualifies (no kernel support).

window_size_right: Option<usize>

Right sliding-window bound — disqualifies (no kernel support).

softcap: Option<f32>

Logit softcap — disqualifies (no kernel support).

k_len: DynScalar

The live attended prefix length (cached_len + seq), resolved via the per-pass SymEnv at realize. For a per-step (non-persistent) decode graph this is DynScalar::Concrete(cached_len + seq); for persistent (plan-once) decode it is a Sym bound to the live length each token.

decomposed_out: NodeId

Arm 0’s exit: the decomposed region’s output node (the oracle).

reconverge: NodeId

The sole consumer of decomposed_out — the branch merge point.

Trait Implementations

impl Clone for DecodeFlashSpec

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for DecodeFlashSpec

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.