Module sampling 

Token sampling from a decoder’s output logits: temperature, top-k, top-p (nucleus), and repetition penalty, on top of the greedy argmax ferrox previously always used unconditionally.

crate::speculative verifies draft tokens against sampling_distribution – the exact distribution Sampler draws from for a given SamplingParams – so speculation is lossless with respect to whatever sampling configuration the caller asked for, rather than only at temperature 0.

No external rand dependency: a small xorshift64* generator (the same algorithm Decoder::new_random_small’s test-only Lcg already uses in decoder.rs) is enough for sampling and keeps the dependency tree the same minimal, pure-Rust shape as the rest of this crate.

Structs

RecommendedSampling

The sampling a checkpoint recommends for itself, one Option per field so that “this model says nothing about top_p” stays distinguishable from “this model recommends top_p = 1.0”. This is sglang’s sampling_defaults='model', ported from FreeToken python/freetoken/utils/hf.py:92 load_generation_sampling.

RequestedSampling

The sampling fields one request actually specified. None means the request said nothing about that field, so the checkpoint’s recommendation (and then the framework default) may speak for it.

Sampler

A small, seedable xorshift64* generator. Not cryptographically secure – sampling doesn’t need that – but reproducible given a seed, which greedy argmax already was for free.

SamplingParams

Sampling parameters for one generation request. temperature <= 0.0 means “sample nothing, take the greedy argmax” – the same deterministic behavior ferrox always had before this module existed.

Functions

sampling_distribution

The exact distribution Sampler::sample draws from for these logits, params and history: penalties applied, temperature divided in, top-k and top-p filtered, renormalised to sum to 1.

Type Aliases

LogitMask

Zeroes out logits a caller wants to forbid, in place, before the sampler looks at them. Used by JSON-object mode to keep generation inside the grammar.