Skip to main content

sampling_distribution

Function sampling_distribution 

Source
pub fn sampling_distribution(
    logits: &[f32],
    params: &SamplingParams,
    history: &[usize],
) -> Vec<f32>
Expand description

The exact distribution Sampler::sample draws from for these logits, params and history: penalties applied over the penalty_last_n window, then top-k, top-p and min-p, then temperature, renormalised to sum to 1.

That is llama.cpp’s chain order, and it is the order filtered_distribution runs – temperature last, not first. This comment used to say “temperature divided in, top-k and top-p filtered”, which described the pre-2026-09-01 pipeline and omitted min-p entirely.

This is what makes lossless speculative verification possible. The speculative-sampling rejection rule compares p_target(x) against the draft’s q(x), and “the target’s probability” is meaningless unless it is the probability the configured sampler would actually have used – a rule that compared against the raw softmax while the server sampled with top_p = 0.9 would be lossless with respect to a model nobody is running.

Greedy (temperature <= 0.0) is a distribution too: the point mass on the argmax. Returning it as one rather than as a special case is why the same verification code is correct at every temperature.