Skip to main content

ngram_simple_draft

Function ngram_simple_draft 

Source
pub fn ngram_simple_draft(
    size_ngram: u16,
    size_mgram: u16,
    tokens: &[LlamaToken],
    sampled: LlamaToken,
) -> Result<Vec<LlamaToken>, NgramError>
Expand description

Draft by finding the most recent repeat of the trailing n-gram.

Looks for the last size_ngram tokens elsewhere in tokens and proposes the size_mgram tokens that followed it. Entirely stateless — the whole strategy is “this text repeated once, it may repeat again”.

tokens is the history excluding sampled: upstream builds its search pattern from the tail of tokens plus sampled, so passing a history that already ends in sampled searches for the wrong thing.

Returns an empty draft when nothing matches, which is the common case for prose and the reason this is nearly free. Also empty when the history is shorter than size_ngram + size_mgram + 1, or when the only match sits at position 0 — upstream treats index 0 as “no match”.

§Errors

Returns NgramError::Failed if llama.cpp throws.