Struct TopPSampler 

pub struct TopPSampler {
    pub p: f64,
    pub temperature: f64,
    /* private fields */
}

Nucleus (top-p) sampler: keeps the smallest set of tokens whose cumulative probability is at least p, then samples from that nucleus.

Fields

p: f64

Cumulative probability threshold in (0, 1].

temperature: f64

Temperature applied before the nucleus filter.

Implementations

impl TopPSampler

pub fn new(p: f64, temperature: f64, seed: u64) -> Result<Self, SamplingError>

Construct a TopPSampler.

Fails if p <= 0.0 || p > 1.0 or if temperature <= 0.0.

pub fn sample(&mut self, logits: &[f64]) -> Result<SampledToken, SamplingError>

Sample one token from logits using nucleus sampling.

pub fn apply_top_p(probs: &[f64], p: f64) -> Vec<f64>

Given a probability vector probs, return a logit vector in which tokens outside the nucleus are set to f64::NEG_INFINITY.

Algorithm:

1. Sort indices by descending probability.
2. Accumulate until the sum >= p.
3. All tokens beyond the cutoff become NEG_INFINITY.

Trait Implementations

impl Debug for TopPSampler

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.