Skip to main content

Module common_sampler

Module common_sampler 

Source
Expand description

llama.cpp’s assembled sampler chain, and the reasoning-budget sampler.

LlamaSampler wraps the individual llama_sampler_* primitives, leaving the caller to build a chain. This wraps common_sampler — the chain upstream assembles for its own tools — which is a different trade: less control, but it gets right several things that are easy to miss when hand-rolling.

  • Ordering. Penalties and DRY run before truncation samplers, which run before temperature. A chain in the wrong order silently samples from the wrong distribution.
  • Grammar prefill. Output-format and tool-call grammars are advanced past the generation prompt; user grammars are not. See ChatParams::grammar_sampler for what goes wrong otherwise.
  • Model-declared suppress tokens are merged into the logit bias, so tokenizer.ggml.suppress_tokens is honoured without the caller knowing it exists.
  • Reasoning budget, created whenever a lazy grammar is active so a thinking block can be force-closed.
  • Speculative acceptance via CommonSampler::sample_and_accept_n.

Structs§

CommonSampler
llama.cpp’s assembled sampler chain.
CommonSamplerParams
Parameters for CommonSampler::new, seeded from llama.cpp’s defaults.
ReasoningBudget
A sampler that caps how long a model may think.

Enums§

CommonSamplerType
Which of llama.cpp’s samplers to run, and in what order.
GrammarSource
How a grammar was obtained, which decides whether the generation prompt is prefilled into it.
ReasoningBudgetState
Where a ReasoningBudget is in its state machine.

Type Aliases§

CommonSamplerError
Errors from the common-sampler layer.
CommonSamplerScalars
The scalar half of llama.cpp’s sampling parameters.