Struct CommonSampler 

pub struct CommonSampler { /* private fields */ }

llama.cpp’s assembled sampler chain.

Implementations

impl CommonSampler

pub fn new( model: &LlamaModel, params: &mut CommonSamplerParams, ) -> Result<Self, ShimError>

Assemble the chain for model.

Errors

Returns CommonSamplerError::Failed if llama.cpp cannot build it — most often a grammar that does not parse.

pub fn sample( &mut self, ctx: &mut LlamaContext<'_>, idx: i32, grammar_first: bool, ) -> Result<LlamaToken, ShimError>

Sample a token from the logits at idx.

grammar_first applies the grammar before the other samplers rather than after. Upstream’s default is false: sample first and only fall back to grammar-constrained resampling if the pick is rejected, which is much cheaper than filtering the whole vocabulary every step.

Errors

Returns CommonSamplerError::Failed if llama.cpp throws.

pub fn accept(&mut self, token: LlamaToken, is_generated: bool)

Feed a token back into the sampler’s history.

is_generated marks a token the model produced, as opposed to one from the prompt; penalties and the grammar treat the two differently.

pub fn sample_and_accept_n( &mut self, ctx: &mut LlamaContext<'_>, draft: &[LlamaToken], grammar_first: bool, ) -> Result<Vec<LlamaToken>, ShimError>

Validate a speculative draft and return the accepted prefix.

This is the acceptance half of speculative decoding: given draft tokens proposed by a drafter and a target-model forward pass covering them, it returns every draft token the target agrees with, plus one freshly sampled token at the first divergence. The result is therefore never empty and never longer than draft.len() + 1.

Accepted tokens are also fed to Self::accept internally, so the caller must not do so again.

Errors

Returns CommonSamplerError::Failed if llama.cpp throws.

pub fn reset(&mut self)

Clear all history and grammar state, keeping the configuration.

pub fn try_clone(&self) -> Result<Self, ShimError>

Deep-copy this sampler, state included.

Errors

Returns CommonSamplerError::Failed if llama.cpp returns null.

pub fn seed(&self) -> u32

The seed actually in use, after any LLAMA_DEFAULT_SEED was resolved.

pub fn last(&self) -> LlamaToken

The most recently sampled token.

pub fn force_end_reasoning(&mut self) -> bool

End the current reasoning block now, as if the budget had run out.

Returns false when there is no budget sampler or it is not counting.

pub fn describe(&self) -> Result<String, ShimError>

Describe the assembled chain, e.g. "penalties -> top_k -> temp".

Errors

Returns CommonSamplerError::Failed if llama.cpp throws.

pub fn prev_str( &mut self, ctx: &mut LlamaContext<'_>, n: i32, ) -> Result<String, ShimError>

Detokenize the last n sampled tokens.

Errors

Returns CommonSamplerError::Failed if llama.cpp throws.

Trait Implementations

impl Debug for CommonSampler

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Drop for CommonSampler

fn drop(&mut self)

Executes the destructor for this type.

fn pin_drop(self: Pin<&mut Self>)

🔬This is a nightly-only experimental API. (pin_ergonomics)

Execute the destructor for this type, but different to Drop::drop, it requires self to be pinned.

impl Send for CommonSampler