Struct NgramMap 

pub struct NgramMap { /* private fields */ }

An adaptive in-context n-gram drafter.

Like NgramCache it indexes repeated n-grams, but it also records how many tokens each of its own drafts got accepted and uses that to decide whether to draft again — so a context where lookup is not paying off stops costing anything. Feed results back with Self::accept.

State lives only in memory and only for this generation; call Self::begin when starting a new one.

Wraps common_ngram_map.

Implementations

impl NgramMap

pub fn new( size_key: u16, size_value: u16, key_only: bool, min_hits: u16, ) -> Result<Self, NgramError>

Build a map.

• size_key — length of the n-grams used as lookup keys.
• size_value — length of the continuations drafted.
• key_only — index keys without tracking continuations, which is cheaper but drafts nothing on its own.
• min_hits — how many times a key must recur before it is trusted.

Errors

Returns NgramError::Failed if llama.cpp cannot allocate the map — it reserves a 2^18-entry hash table up front.

pub fn begin(&mut self, tokens: &[LlamaToken]) -> Result<(), NgramError>

Start a generation over tokens (the prompt).

Errors

Returns NgramError::Failed if llama.cpp throws.

pub fn draft( &mut self, tokens: &[LlamaToken], sampled: LlamaToken, ) -> Result<Vec<LlamaToken>, NgramError>

Draft a continuation, given everything generated so far and the token just sampled.

Returns an empty draft when the map decides lookup is not worth it here.

Errors

Returns NgramError::Failed if llama.cpp throws.

pub fn accept(&mut self, n_accepted: u16)

Report how many of the last draft’s tokens the target model accepted.

This is what makes the map adaptive; skipping it leaves it drafting blind.

Trait Implementations

impl Debug for NgramMap

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Drop for NgramMap

fn drop(&mut self)

Executes the destructor for this type.

fn pin_drop(self: Pin<&mut Self>)

🔬This is a nightly-only experimental API. (pin_ergonomics)

Execute the destructor for this type, but different to Drop::drop, it requires self to be pinned.

impl Send for NgramMap