pub struct CommonSamplerParams { /* private fields */ }Expand description
Parameters for CommonSampler::new, seeded from llama.cpp’s defaults.
The scalar knobs are a plain public struct (CommonSamplerScalars);
everything backed by a container is set through a method, which keeps this
wrapper stable when upstream adds fields.
Implementations§
Source§impl CommonSamplerParams
impl CommonSamplerParams
Sourcepub fn new() -> Self
pub fn new() -> Self
Allocate with llama.cpp’s defaults (temp 0.8, top-k 40, top-p 0.95, …).
§Panics
Panics if the allocation fails.
Sourcepub fn scalars(&self) -> CommonSamplerScalars
pub fn scalars(&self) -> CommonSamplerScalars
Read the scalar knobs.
Sourcepub fn set_scalars(&mut self, scalars: &CommonSamplerScalars)
pub fn set_scalars(&mut self, scalars: &CommonSamplerScalars)
Replace the scalar knobs. Read Self::scalars first and modify it, so
fields you do not care about keep llama.cpp’s defaults.
Sourcepub fn set_grammar(
&mut self,
grammar: &str,
source: GrammarSource,
lazy: bool,
) -> Result<(), ShimError>
pub fn set_grammar( &mut self, grammar: &str, source: GrammarSource, lazy: bool, ) -> Result<(), ShimError>
Constrain sampling with a GBNF grammar.
source decides whether the generation prompt is prefilled — see
GrammarSource.
§Errors
Returns CommonSamplerError::Nul for an interior NUL in grammar.
Sourcepub fn add_grammar_trigger(
&mut self,
kind: &str,
value: &str,
token: LlamaToken,
) -> Result<(), ShimError>
pub fn add_grammar_trigger( &mut self, kind: &str, value: &str, token: LlamaToken, ) -> Result<(), ShimError>
Add a trigger that activates a lazy grammar.
kind matches GrammarTrigger::kind:
"token", "word", "pattern" or "pattern_full".
§Errors
Returns CommonSamplerError::InvalidArg for an unknown kind, or
CommonSamplerError::Nul for an interior NUL.
Sourcepub fn set_generation_prompt(&mut self, prompt: &str) -> Result<(), ShimError>
pub fn set_generation_prompt(&mut self, prompt: &str) -> Result<(), ShimError>
Set the generation prompt the grammar should be advanced past.
§Errors
Returns CommonSamplerError::Nul for an interior NUL.
Sourcepub fn add_logit_bias(
&mut self,
token: LlamaToken,
bias: f32,
) -> Result<(), ShimError>
pub fn add_logit_bias( &mut self, token: LlamaToken, bias: f32, ) -> Result<(), ShimError>
Bias a token’s logit. Use f32::NEG_INFINITY to ban it outright.
§Errors
Returns CommonSamplerError::InvalidArg if the params handle is bad.
Sourcepub fn set_samplers(
&mut self,
samplers: &[CommonSamplerType],
) -> Result<(), ShimError>
pub fn set_samplers( &mut self, samplers: &[CommonSamplerType], ) -> Result<(), ShimError>
Replace the sampler ordering.
§Errors
Returns CommonSamplerError::InvalidArg if the params handle is bad.
Sourcepub fn set_dry_breakers(&mut self, breakers: &[&str]) -> Result<(), ShimError>
pub fn set_dry_breakers(&mut self, breakers: &[&str]) -> Result<(), ShimError>
Replace the DRY sequence breakers (default: newline, :, ", *).
§Errors
Returns CommonSamplerError::Nul for an interior NUL.
Sourcepub fn set_reasoning_budget(
&mut self,
start: &[LlamaToken],
ends: &[Vec<LlamaToken>],
forced: &[LlamaToken],
message: &str,
) -> Result<(), ShimError>
pub fn set_reasoning_budget( &mut self, start: &[LlamaToken], ends: &[Vec<LlamaToken>], forced: &[LlamaToken], message: &str, ) -> Result<(), ShimError>
Cap how many tokens the model may spend inside a reasoning block.
start is the tokenized opening tag (<think>), ends the closing
tags — the first doubles as the forcing sequence. forced is emitted
when the budget runs out, typically a truncation message followed by the
closing tag. Set the budget itself through
CommonSamplerScalars::reasoning_budget_tokens.
§Errors
Returns CommonSamplerError::Nul for an interior NUL in message.