Struct PlanRuntimeDecodeInput 

pub struct PlanRuntimeDecodeInput {
    pub request_id: RequestId,
    pub input_token: TokenId,
    pub kv_cache: Arc<dyn KvCacheHandle>,
    pub logits_policy: LogitsReturnPolicy,
}

Tensor-free decode input for an executor with plan-runtime resource authority.

The runtime already owns the request’s physical cache and accepts exactly one token per decode frontier, so materializing a host tensor here adds no information. Legacy and modality executors continue to use DecodeInput.

Fields

request_id: RequestId

input_token: TokenId

kv_cache: Arc<dyn KvCacheHandle>

logits_policy: LogitsReturnPolicy

Implementations

impl PlanRuntimeDecodeInput

pub fn new( request_id: RequestId, input_token: TokenId, kv_cache: Arc<dyn KvCacheHandle>, ) -> Self

pub fn with_logits_policy(self, logits_policy: LogitsReturnPolicy) -> Self

Trait Implementations

impl Clone for PlanRuntimeDecodeInput

fn clone(&self) -> PlanRuntimeDecodeInput

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for PlanRuntimeDecodeInput

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.