Enum PendingTextInput 

pub enum PendingTextInput<P, T> {
    Prefill(P),
    Decode(T),
}

Input for the next ordinary prediction. A committed token may still be the pending decode input and must not be inserted into model state until that step.

Variants

Prefill(P)

The original prepared prompt, before the first model prediction.

Decode(T)

The preceding committed canonical token, before its successor is predicted.

Implementations

impl<P, T> PendingTextInput<P, T>

pub fn as_ref(&self) -> PendingTextInput<&P, &T>

Borrows the pending native input without cloning or evaluating it.