Struct LlamaSampler 

pub struct LlamaSampler { /* private fields */ }

A sampler: an ordered chain of transform stages ending in a selector, matching llama-cpp-2’s LlamaSampler.

Build single stages with the constructors (greedy, temp, top_k, …) and compose them with chain_simple. During generation, either call sample (which reads logits from the context) or apply on a candidate array you already built, then read LlamaTokenDataArray::selected_token. Call accept with each drawn token so stateful stages (penalties, grammar) stay in sync.

Not Clone (a grammar stage owns a C grammar object freed on drop).

Implementations

impl LlamaSampler

pub fn chain_simple(samplers: impl IntoIterator<Item = LlamaSampler>) -> Self

Compose several samplers into one chain, applied left to right.

pub fn greedy() -> Self

Greedy (argmax) selector.

pub fn dist(seed: u32) -> Self

Stochastic distribution selector seeded by seed (softmax + draw).

pub fn temp(t: f32) -> Self

Temperature scaling.

pub fn temp_ext(t: f32, delta: f32, exponent: f32) -> Self

Dynamic-temperature (“entropy”) scaling over [t-delta, t+delta].

pub fn top_k(k: i32) -> Self

Top-k filtering.

pub fn top_p(p: f32, min_keep: usize) -> Self

Top-p (nucleus) filtering.

pub fn min_p(p: f32, min_keep: usize) -> Self

Min-p filtering.

pub fn typical(p: f32, min_keep: usize) -> Self

Locally-typical filtering.

pub fn top_n_sigma(n: f32) -> Self

Top-nσ filtering.

pub fn tail_free(z: f32, min_keep: usize) -> Self

Tail-free filtering.

pub fn grammar( model: &LlamaModel, grammar_str: &str, root: &str, ) -> Result<Self, GrammarInitError>

A GBNF grammar constraint stage.

grammar_str is GBNF source, root the start symbol (usually "root"). The stage masks tokens the grammar cannot currently accept; call accept with each drawn token to advance grammar state.

Safety contract

The returned sampler stashes a pointer into model’s vocab, so model must outlive this sampler — using it after the model is dropped is undefined behavior. This is not encoded in the type (the sampler stays 'static) so that it drops in for llama-cpp-2, which makes the same choice; keep the model alive at least as long as any sampler built from it (a 'static model trivially satisfies this). Prefer crate::grammar::LlamaGrammar if you want the model lifetime enforced.

Errors

GrammarInitError::Nul on an interior NUL byte; GrammarInitError::Parse if the GBNF fails to parse.

pub fn grammar_lazy( model: &LlamaModel, grammar_str: &str, root: &str, trigger_words: impl IntoIterator<Item = impl AsRef<[u8]>>, trigger_tokens: &[LlamaToken], ) -> Result<Self, GrammarInitError>

A lazy GBNF grammar constraint: it stays inert until one of trigger_words / trigger_tokens appears, then constrains generation.

Errors

GrammarInitError::Nul on an interior NUL byte in the grammar, root, or a trigger word; GrammarInitError::Parse if the GBNF fails to parse.

pub fn penalties( penalty_last_n: i32, penalty_repeat: f32, penalty_freq: f32, penalty_present: f32, ) -> Self

Repetition / frequency / presence penalties over the accepted history.

pub fn apply(&mut self, arr: &mut LlamaTokenDataArray)

Run every stage over arr in order (transforms + selector). Read the result with LlamaTokenDataArray::selected_token.

A no-op on an empty candidate array (leaving selected = None): ik’s legacy samplers GGML_ASSERT(size > 0) and would abort the process, so this safe wrapper must not forward an empty set into them.

pub fn accept(&mut self, token: LlamaToken)

Record token: appends to the history (penalties stage) and advances any grammar stage.

If a grammar stage is present, only feed tokens the grammar permits at the current position (i.e. tokens surviving apply). Accepting a grammar-disallowed end-of-generation token aborts the process (ik’s llama_grammar_accept_impl treats it as fatal).

pub fn reset(&mut self)

Clear the accepted-token history.

pub fn sample(&mut self, ctx: &LlamaContext<'_>, idx: i32) -> LlamaToken

Build the candidate array from the context’s logits at batch index idx, run the chain, and return the selected token (argmax fallback if no selector ran).

Trait Implementations

impl Debug for LlamaSampler

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.