Struct MtpSpeculative 

pub struct MtpSpeculative<'model> { /* private fields */ }

Full MTP speculative driver (requires the common feature).

Wraps ik’s common_speculative_* via the ik_llama_rs_mtp_* C++ glue. Two ways to drive it after new → begin(prompt):

• step() — the convenience path: the glue owns the whole draft→verify→sample→commit cycle (greedy / temp), returning the committed tokens. Use this when you don’t need to constrain sampling.
• draft() + commit() — the caller-driven path: draft returns the NextN candidates without sampling; you build the verify batch [id_last] + drafts (logits on every position), decode it on target_context_mut, pick each committed token yourself (e.g. through a crate::LlamaGrammar + sampler over target_context’s logits), then commit them. This is what lets MTP speculation run together with grammar- constrained / custom sampling — the crate never samples on this path.

Either way ik owns all KV bookkeeping (the companion cache and the target rollback happen inside the glue); the caller must not clear or roll back either cache itself.

Implementations

impl<'model> MtpSpeculative<'model>

pub fn new( model: &LlamaModel, ctx: LlamaContext<'model>, params: MtpSpeculativeParams, ) -> Result<Self, LlamaError>

Initialize the driver over a NextN model + its (mtp-enabled) context.

Takes ownership of ctx (so the driver is self-contained and can be stored in a struct); read/decode it back out via target_context / target_context_mut.

model must have been loaded with .with_mtp(true) and ctx created with .with_mtp(true). Fails with LlamaError::MtpInit for a model with 0 NextN layers (not an MTP model). Recurrent / openPangu targets (which includes the Qwen3.5 NextN family) are supported: the driver takes an internal rollback checkpoint before each verify so rejected drafts restore the recurrent state correctly.

pub fn begin(&mut self, prompt: &[LlamaToken]) -> Result<usize, LlamaError>

Prompt warmup + begin. Runs the prompt through the MTP path and prepares the first draft. Returns n_past. Call once before Self::step.

pub fn step(&mut self) -> Result<MtpStep, LlamaError>

Run one draft→verify→accept→commit cycle. Returns the tokens emitted this step (to append to the output) and how many drafts were accepted.

pub fn draft( &mut self, n_past: i32, id_last: LlamaToken, ) -> Result<Vec<LlamaToken>, LlamaError>

Propose up to n_max draft tokens following id_last (the last committed token) at position n_past, without sampling.

This is the caller-driven entry point for running MTP speculation alongside a grammar / custom sampler (see the module example). After draft, the caller must:

1. build the verify batch [id_last] + drafts (one sequence, logits on every position), in that order;
2. decode it on target_context_mut;
3. pick the committed tokens with its own grammar/sampler, reading output index i via target_context for each verify position i;
4. call commit.

Use one round strictly as draft → verify-decode → commit: commit takes its anchor position from the n_past you pass here, so do not draft twice before committing, and build the verify batch with id_last at exactly n_past. Do not mix this path with step on the same driver — they own the loop state differently.

Errors

LlamaError::MtpStep if the C glue rejects the draft.

pub fn commit( &mut self, id_last: LlamaToken, committed: &[LlamaToken], n_draft: usize, ) -> Result<(), LlamaError>

Finalize one speculative round after the caller has decoded the verify batch and chosen its committed tokens.

id_last is the anchor token that led the verify batch (the same one passed to draft); committed is the accepted-draft prefix followed by exactly one correction/bonus token (so committed.len() == n_accepted + 1); n_draft is the number of tokens the matching draft returned.

Advances the MTP companion by the accepted count and rolls the target KV cache back to the committed prefix. The caller must not clear or roll back either KV cache itself — ik owns that bookkeeping.

Errors

LlamaError::MtpStep if committed is empty, longer than n_draft + 1 (more tokens than the verify batch produced — which would desync the KV cache), or the glue rejects the commit.

pub fn params(&self) -> MtpSpeculativeParams

The configured parameters.

pub fn target_context(&self) -> &LlamaContext<'model>

Shared access to the target context — read per-position logits here for a grammar-gated commit (e.g. LlamaContext::token_data_array_ith).

pub fn target_context_mut(&mut self) -> &mut LlamaContext<'model>

Mutable access to the target context — decode the verify batch here.

pub fn context_mut(&mut self) -> &mut LlamaContext<'model>

Mutable access to the wrapped context (only between full generations).

Alias of target_context_mut, kept for back-compat.

Trait Implementations

impl<'model> Debug for MtpSpeculative<'model>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Drop for MtpSpeculative<'_>

fn drop(&mut self)

Executes the destructor for this type.

fn pin_drop(self: Pin<&mut Self>)

🔬This is a nightly-only experimental API. (pin_ergonomics)

Execute the destructor for this type, but different to Drop::drop, it requires self to be pinned.