Struct NgramCache 

pub struct NgramCache { /* private fields */ }

Token-level n-gram cache for speculative draft generation.

Records bigram and trigram patterns from generated text and predicts likely next tokens based on observed frequencies.

Implementations

impl NgramCache

pub fn new() -> Self

Create a new empty n-gram cache.

pub fn record(&mut self, tokens: &[u32])

Record a sequence of tokens into the cache.

Updates both bigram and trigram frequency tables.

pub fn predict_one(&self, context: &[u32]) -> Option<u32>

Predict the most likely next token given the context.

Tries trigram first (higher accuracy), falls back to bigram. Returns None if no matching pattern is found.

pub fn draft(&self, context: &[u32], lookahead: usize) -> Vec<u32>

Predict up to lookahead tokens by chaining predictions.

Each predicted token is appended to the context for the next prediction. Stops early if no prediction is available.

pub fn trigram_count(&self) -> usize

Number of unique trigram keys stored.

pub fn bigram_count(&self) -> usize

Number of unique bigram keys stored.

pub fn is_empty(&self) -> bool

Returns true if the cache has no entries.

Trait Implementations

impl Default for NgramCache

fn default() -> Self

Returns the “default value” for a type.