Module ngram 

Speculative decoding with no draft model.

Eagle3Session and MtpSession draft with a second model. These draft by looking up what came before: find where the recent token history repeats, and propose whatever followed it last time. That costs one hash lookup per step instead of a forward pass, needs no extra weights and no extra VRAM, and wins big on the workloads where text repeats — code editing, RAG over a quoted document, JSON with recurring keys, chat that restates the question.

Three strategies, in increasing order of memory and payoff:

	Learns from	State
ngram_simple_draft	the current context only	none
NgramMap	the current context, adapting to how its drafts land	in memory
NgramCache	a corpus and/or past generations	in memory, saveable

All three return a draft: candidate tokens to verify against the target model, typically via CommonSampler::sample_and_accept_n. A wrong draft costs only the tokens it wasted.

Structs

NgramCache

A statistical n-gram cache: which tokens tend to follow which n-grams.

NgramMap

An adaptive in-context n-gram drafter.

Functions

ngram_cache_draft

Draft a continuation from up to three caches.

ngram_simple_draft

Draft by finding the most recent repeat of the trailing n-gram.

Type Aliases

NgramError

Errors from the n-gram drafters.