Enum PagedDecodePlan 

pub enum PagedDecodePlan {
    Replan,
    PlanOnce,
}

Runtime policy for the paged persistent decode path (LlamaModel::forward_paged_step_persistent): whether to build the plan ONCE and reuse it across tokens, or re-plan every token (the pre-plan-once behavior). The paged driver holds this per config and passes it each step; the driver default is PlanOnce (set by PagedSessionScheduler::new), with Replan available as an explicit opt-out.

The A/B rollout this flag was built for is DONE, and the default moved with the result. Measured (nsys, Lightbulb, 2026-08-01; 16 tokens, same binary and model, only the plan mode differing): Replan → PlanOnce is 29.7× on ms/token (8,192.0 → 275.8) and cuts host-ward DtoH 61% (101,691 → 40,088 MB) at IDENTICAL kernel-launch count (17,802 in both arms — same compute, less data movement). Shipping Replan as the default meant every consumer who did not know to opt in paid that 29.7×.

It remains a real runtime flag — not a test hook — because toggling it in ONE process is what lets the correctness gate run both arms off a single setup, and because Replan is the parity reference the plan-once path is checked against. On Replan the persistent forward drops any held session (so nothing stale lingers) and delegates to the re-planning LlamaModel::forward_paged_step; on PlanOnce it builds-once / rebinds.

Variants

Replan

Re-plan (build + optimize the graph) every token — the pre-plan-once path. Drops any held PagedDecodeSession and routes to LlamaModel::forward_paged_step.

PlanOnce

Build + optimize the graph ONCE (into a PagedDecodeSession) and reuse it (rebinding only the per-token data) for every subsequent token.

Trait Implementations

impl Clone for PagedDecodePlan

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for PagedDecodePlan

impl Debug for PagedDecodePlan

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for PagedDecodePlan

impl PartialEq for PagedDecodePlan

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for PagedDecodePlan