Skip to main content

decode_batch_unconverted

Function decode_batch_unconverted 

Source
pub fn decode_batch_unconverted(plan: &ModelPlan) -> bool
Expand description

Does this plan’s residual topology have NO converted batched-decode arm?

HyperConnections carries streams parallel residual streams collapsed per layer; every batched path (decode_step_batch*, prime_cache_batch, prime_graph_new, the graph-capture and speculative entry points) runs a serial residual and refuses this topology at the engine boundary rather than computing a different model. The converted set is forward, forward_last, prime_cache, decode_step — all reachable from the per-session eager route.

This is the plan-level twin of the Gemma arm of decode_batch_program: the scheduler needs to know a model has no batched arm BEFORE it places the session, so the topology is read from the ModelPlan rather than re-derived from engine internals.