Struct Sampler 

pub struct Sampler { /* private fields */ }

A small, seedable xorshift64* generator. Not cryptographically secure – sampling doesn’t need that – but reproducible given a seed, which greedy argmax already was for free.

Implementations

impl Sampler

pub fn new(seed: u64) -> Self

pub fn xtc_roll(&mut self, params: &SamplingParams) -> Option<f32>

The one uniform draw XTC needs for this token, or None when XTC cannot fire for these parameters.

XTC is the only sampler in the chain that is itself stochastic (llama_sample_xtc_apply draws from its own std::mt19937, src/llama-sampler.cpp:2146), which is why the chain below takes the roll as an argument instead of owning an RNG: the chain is also what sampling_distribution runs for speculative verification, and a filter that drew its own randomness there would make “the distribution the sampler draws from” a different distribution every time it was asked.

The draw is skipped when XTC cannot fire, and that is not an optimisation. Every draw advances the seeded stream, so drawing unconditionally would shift every subsequent token of every existing seeded generation, on every run that never asked for XTC. SamplingParams::xtc_can_fire is the single predicate this and [Candidates::xtc] share.

pub fn sample( &mut self, logits: &[f32], params: &SamplingParams, history: PenaltyWindow<'_>, ) -> usize

Samples one token id from logits, given params and the PenaltyWindow the penalties look back over. Falls back to plain greedy argmax when params.temperature <= 0.0.

history is a window and not a slice on purpose: it carries the PROMPT as well as the generated tokens, which is what llama.cpp penalises over. See crate::penalty_window.

A length-1 logits vector is treated as a precomputed greedy token id (logits[0] as usize) — used by the Metal dense-stack path that returns GPU argmax instead of downloading the full vocab.

pub fn sample_with_mask( &mut self, logits: &[f32], params: &SamplingParams, history: PenaltyWindow<'_>, mask: Option<LogitMask<'_>>, ) -> usize

Like Self::sample, but optionally zeroes disallowed logits via mask before argmax / nucleus sampling (used for JSON-object mode).

pub fn sample_reporting( &mut self, logits: &[f32], params: &SamplingParams, history: PenaltyWindow<'_>, mask: Option<LogitMask<'_>>, ) -> (usize, Option<Vec<f32>>)

The token AND the distribution it was drawn from, normalised to sum to 1 over the whole vocabulary.

This is what logprobs has to report: not the raw logits, but the distribution the sampler actually drew from, with the penalties applied over the penalty_last_n window and llama.cpp’s chain run in params.sampler_order. A filtered-out candidate is a zero, which is what “this token could not have been chosen” means.

None for a vocabulary this sampler never saw: a backend that folded lm_head and argmax onto the device hands back a one-element vector holding the chosen id, and there is no distribution to report for it. A caller that needs one must ask for the vocabulary (GenerationParams::needs_vocab_logits) rather than be given a fabricated single-candidate answer.

It is the SAME vector Self::sample_with_mask draws from – both go through sample_inner – so a reported logprob cannot describe a distribution other than the one that was sampled.

Not sampling_distribution, and the difference is the point. That function recomputes the pipeline from the logits WITHOUT drawing, which is right for speculative verification (it needs p_target(x) for a token someone else proposed) and wrong here for two reasons: it takes xtc_roll as an argument, so a caller who passed a fresh roll would report a distribution the draw never saw; and for a greedy request it returns a ONE-HOT, which as a logprob would claim the model was certain when nobody asked it. This reports the real distribution in both cases, because “how confident was the model” is a question a greedy caller is entitled to ask.

pub fn uniform(&mut self) -> f32

A uniform draw in [0.0, 1.0).

Exposed because speculative decoding’s accept test is a coin flip against p_target(x) / p_draft(x) rather than a draw from a distribution, and it must come off the same seeded stream as every other draw in the run or a “reproducible given a seed” generation stops being reproducible.

pub fn sample_from(&mut self, probs: &[f32]) -> usize

Draws one index from an already-normalised distribution.

Split out of Self::sample_with_mask so speculative decoding can sample from a distribution it had to compute anyway (the rejection rule needs p_target itself, not just a draw from it) and still go through exactly the same draw as ordinary sampling. Two separate copies of this loop would be two chances to be subtly non-lossless.