Expand description
llama.cpp’s assembled sampler chain, and the reasoning-budget sampler.
LlamaSampler wraps the individual
llama_sampler_* primitives, leaving the caller to build a chain. This
wraps common_sampler — the chain upstream assembles for its own tools —
which is a different trade: less control, but it gets right several things
that are easy to miss when hand-rolling.
- Ordering. Penalties and DRY run before truncation samplers, which run before temperature. A chain in the wrong order silently samples from the wrong distribution.
- Grammar prefill. Output-format and tool-call grammars are advanced
past the generation prompt; user grammars are not. See
ChatParams::grammar_samplerfor what goes wrong otherwise. - Model-declared suppress tokens are merged into the logit bias, so
tokenizer.ggml.suppress_tokensis honoured without the caller knowing it exists. - Reasoning budget, created whenever a lazy grammar is active so a thinking block can be force-closed.
- Speculative acceptance via
CommonSampler::sample_and_accept_n.
Structs§
- Common
Sampler - llama.cpp’s assembled sampler chain.
- Common
Sampler Params - Parameters for
CommonSampler::new, seeded from llama.cpp’s defaults. - Reasoning
Budget - A sampler that caps how long a model may think.
Enums§
- Common
Sampler Type - Which of llama.cpp’s samplers to run, and in what order.
- Grammar
Source - How a grammar was obtained, which decides whether the generation prompt is prefilled into it.
- Reasoning
Budget State - Where a
ReasoningBudgetis in its state machine.
Type Aliases§
- Common
Sampler Error - Errors from the common-sampler layer.
- Common
Sampler Scalars - The scalar half of llama.cpp’s sampling parameters.