Struct MtpSpeculative 

pub struct MtpSpeculative<'model> { /* private fields */ }

RAII owner for a same-model MTP speculative context.

This wrapper currently binds llama.cpp’s speculative state to sequence 0. Batches passed to Self::process must therefore contain only sequence 0.

Implementations

impl<'model> MtpSpeculative<'model>

pub fn new( target_context: LlamaContext<'model>, draft_context: LlamaContext<'model>, params: MtpSpeculativeParams, ) -> Result<Self, MtpSpeculativeError>

Create a new MTP speculative helper from a target context and an MTP draft context.

Errors

Returns an error if parameters are invalid or llama.cpp cannot initialize the speculative implementation for the loaded model.

pub fn target_context(&self) -> &LlamaContext<'model>

Access the target context.

pub fn target_context_mut(&mut self) -> &mut LlamaContext<'model>

Access the target context for decode and cache rollback operations.

pub fn draft_context_mut(&mut self) -> &mut LlamaContext<'model>

Access the draft context for cache rollback operations.

pub fn begin( &mut self, prompt_tokens: &[LlamaToken], ) -> Result<(), MtpSpeculativeError>

Begin a new generation from the given prompt tokens.

Errors

Returns an error if llama.cpp rejects the call.

pub fn process( &mut self, batch: &LlamaBatch<'_>, ) -> Result<(), MtpSpeculativeError>

Process a batch that was just decoded by the target context.

The batch must contain token input for sequence 0 only.

Errors

Returns an error if llama.cpp cannot update the MTP draft context.

pub fn draft( &mut self, n_past: i32, id_last: LlamaToken, prompt_tokens: &[LlamaToken], ) -> Result<Vec<LlamaToken>, MtpSpeculativeError>

Generate draft tokens after id_last.

Errors

Returns an error if llama.cpp rejects the draft operation or emits more draft tokens than requested.

pub fn accept(&mut self, n_accepted: u16) -> Result<(), MtpSpeculativeError>

Notify llama.cpp how many draft tokens the target context accepted.

Errors

Returns an error if llama.cpp rejects the call.

Trait Implementations

impl<'model> Debug for MtpSpeculative<'model>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Drop for MtpSpeculative<'_>

fn drop(&mut self)

Executes the destructor for this type.

fn pin_drop(self: Pin<&mut Self>)

🔬This is a nightly-only experimental API. (pin_ergonomics)

Execute the destructor for this type, but different to Drop::drop, it requires self to be pinned.