Skip to main content

Module execution_plan

Module execution_plan 

Source
Expand description

Load-time execution / memory plans (llama.cpp graph-params analogue).

Selected once per model (and cached by batch geometry for decode vs prefill). Hot-path forward never re-derives architecture strings or fused-op availability.

Structsยง

ExecutionPlan
Per-model execution plan: everything the forward path needs that is constant across tokens.
FusedOpCaps
Backend fused-op availability discovered at load / first probe.
MemoryPlan
Memory layout chosen once from the architecture profile.
PlanGeometry
Cache key for decode/prefill plan reuse (batch geometry only).