Struct PromptLookupSpeculator 

pub struct PromptLookupSpeculator {
    pub ngram_size: usize,
    pub max_draft_len: usize,
}

Proposes candidate continuation tokens by looking for the longest available match of the most recent ngram_size tokens earlier in history, and returning up to max_draft_len tokens that followed that earlier occurrence. Returns an empty block if no match is found or history is too short to contain one.

This is deliberately simple (last-match-wins, not best-match or a frequency-weighted choice): the whole point of prompt-lookup decoding is that it’s nearly free to compute, since a wrong guess costs nothing but a rejected batch position, not a correctness bug.

Fields

ngram_size: usize

max_draft_len: usize

Implementations

impl PromptLookupSpeculator

pub fn new(ngram_size: usize, max_draft_len: usize) -> Self

pub fn propose_tokens(&self, history: &[usize]) -> Vec<usize>

Looks for the most recent earlier occurrence of history’s last ngram_size tokens, scanning from the end backwards so the most recent match wins (most likely to reflect current context, e.g. a loop the model is currently in). Returns the tokens that followed that occurrence, truncated to max_draft_len.

Trait Implementations

impl Clone for PromptLookupSpeculator

fn clone(&self) -> PromptLookupSpeculator

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for PromptLookupSpeculator

impl Debug for PromptLookupSpeculator

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Drafter for PromptLookupSpeculator

fn propose( &self, history: &[usize], _target_hidden: &[f32], max_len: usize, ) -> DraftBlock

Proposes at most max_len tokens to follow history. target_hidden is the target model’s final-layer hidden state for history’s last token, or empty when none is available yet (which a drafter that needs it must handle by proposing nothing).