Skip to main content

sample_next

Function sample_next 

Source
pub fn sample_next(
    logits: &[f32],
    history: &[u32],
    opts: &SampleOpts,
    rng: &mut u32,
) -> u32
Expand description

Sample one token from logits according to opts.

Greedy when opts.temperature == 0. Otherwise applies temperature, optional top-k, top-p (nucleus), and a repetition penalty against the history ids (set empty to disable).

rng is a 32-bit Lehmer LCG seed; pass any non-zero value. Returns the new seed so callers can chain.