Skip to main content

wgpu_graph_on

Function wgpu_graph_on 

Source
pub fn wgpu_graph_on(phase: GraphPhase) -> bool
Expand description

The one place that decides whether the whole-token graph runs.

CMF_GPU_WGPU_GRAPH: 0 off everywhere, prefill only for the prompt, anything else on everywhere. Unset: desktop-class GPUs take it for both phases; phone-class UMA takes it for PREFILL only, which is the measurement above rather than a guess — the per-op path keeps decode, where it is seventeen times better.