pub fn sampling_distribution(
logits: &[f32],
params: &SamplingParams,
history: PenaltyWindow<'_>,
xtc_roll: Option<f32>,
) -> Vec<f32>Expand description
The exact distribution Sampler::sample draws from for these
logits, params and history: penalties applied over the
penalty_last_n window, then llama.cpp’s chain in
params.sampler_order, renormalised to sum to 1.
That is llama.cpp’s chain order – temperature last, not first. This comment used to say “temperature divided in, top-k and top-p filtered”, which described the pre-2026-09-01 pipeline and omitted min-p entirely.
This is what makes lossless speculative verification possible. The
speculative-sampling rejection rule compares p_target(x) against
the draft’s q(x), and “the target’s probability” is meaningless
unless it is the probability the configured sampler would actually
have used – a rule that compared against the raw softmax while the
server sampled with top_p = 0.9 would be lossless with respect to
a model nobody is running.
Greedy (temperature <= 0.0) is a distribution too: the point mass
on the token [greedy_choice] would pick. Returning it as one rather
than as a special case is why the same verification code is correct
at every temperature.
xtc_roll is Sampler::xtc_roll’s answer, and it is a REQUIRED
argument rather than something this function draws or defaults,
because XTC is stochastic and the caller owns the seeded stream. A
caller that passes None while XTC is configured gets a chain with
no XTC in it, which is why every caller in this workspace obtains it
from Sampler::xtc_roll and not by writing None.