Skip to main content

sampling_distribution

Function sampling_distribution 

Source
pub fn sampling_distribution(
    logits: &[f32],
    params: &SamplingParams,
    history: PenaltyWindow<'_>,
    xtc_roll: Option<f32>,
) -> Vec<f32>
Expand description

The exact distribution Sampler::sample draws from for these logits, params and history: penalties applied over the penalty_last_n window, then llama.cpp’s chain in params.sampler_order, renormalised to sum to 1.

That is llama.cpp’s chain order – temperature last, not first. This comment used to say “temperature divided in, top-k and top-p filtered”, which described the pre-2026-09-01 pipeline and omitted min-p entirely.

This is what makes lossless speculative verification possible. The speculative-sampling rejection rule compares p_target(x) against the draft’s q(x), and “the target’s probability” is meaningless unless it is the probability the configured sampler would actually have used – a rule that compared against the raw softmax while the server sampled with top_p = 0.9 would be lossless with respect to a model nobody is running.

Greedy (temperature <= 0.0) is a distribution too: the point mass on the token [greedy_choice] would pick. Returning it as one rather than as a special case is why the same verification code is correct at every temperature.

xtc_roll is Sampler::xtc_roll’s answer, and it is a REQUIRED argument rather than something this function draws or defaults, because XTC is stochastic and the caller owns the seeded stream. A caller that passes None while XTC is configured gets a chain with no XTC in it, which is why every caller in this workspace obtains it from Sampler::xtc_roll and not by writing None.