Struct SamplerConfig 

pub struct SamplerConfig {

    pub temperature: f32,
    pub top_k: usize,
    pub top_p: f32,
    pub min_p: f32,
    pub repetition_penalty: f32,
    pub repetition_penalty_window: usize,
    pub seed: Option<u64>,
    pub mirostat: u8,
    pub mirostat_tau: f32,
    pub mirostat_eta: f32,
    pub grammar: Option<Arc<Grammar>>,
    pub token_vocab: Option<Arc<Vec<(u32, Vec<u8>)>>>,
    pub logit_bias: HashMap<u32, f32>,
    pub banned_tokens: Vec<u32>,
    pub dry_multiplier: f32,
    pub dry_base: f32,
    pub dry_allowed_length: usize,
    pub xtc_threshold: f32,
    pub xtc_probability: f32,
    pub typical_p: f32,
    pub top_a: f32,
    pub eta_cutoff: f32,
    pub epsilon_cutoff: f32,
}

Configuration for the sampling strategy.

Fields

temperature: f32

Temperature for logit scaling (1.0 = no scaling, 0.0 = greedy).

top_k: usize

Top-K: only consider the K most likely tokens (0 = disabled).

top_p: f32

Top-P (nucleus): only consider tokens with cumulative probability <= p.

min_p: f32

Min-P: minimum probability threshold relative to the top token.

repetition_penalty: f32

Repetition penalty factor (1.0 = no penalty).

repetition_penalty_window: usize

Number of recent tokens to consider for repetition penalty.

seed: Option<u64>

Random seed for reproducible sampling (None = random).

mirostat: u8

Mirostat mode: 0 = disabled, 2 = Mirostat v2.

mirostat_tau: f32

Mirostat target surprise (tau). Controls coherence vs diversity. Lower = more coherent, higher = more diverse. Default: 5.0.

mirostat_eta: f32

Mirostat learning rate (eta). How fast the algorithm adapts. Default: 0.1.

grammar: Option<Arc<Grammar>>

Optional GBNF grammar for constrained sampling. Logits for tokens that cannot advance the grammar are set to -∞. Skipped during serialization (not representable as JSON directly).

token_vocab: Option<Arc<Vec<(u32, Vec<u8>)>>>

Pre-computed vocabulary (token_id, byte_repr) table used for grammar masking. Must be set when grammar is Some. Build via TokenizerBridge::vocab_bytes().

logit_bias: HashMap<u32, f32>

Per-token logit biases applied before top-k/top-p.

Positive values increase a token’s probability; negative values decrease it. For example, logit_bias[token_id] = 5.0 strongly encourages that token, while -100.0 effectively bans it (use banned_tokens for strict banning).

Applied as: logits[token_id] += bias before the greedy / sampling steps.

banned_tokens: Vec<u32>

Tokens that must never be generated.

Their logits are set to f32::NEG_INFINITY before any other sampling step, including top-k/p filtering. This is a hard constraint — unlike a large negative logit_bias, a banned token will never be selected even if it is the only remaining candidate.

dry_multiplier: f32

DRY penalty multiplier (0.0 = disabled).

Penalises tokens that would continue an n-gram already present in the recent context. Higher values apply stronger penalties.

dry_base: f32

DRY exponential base for match-length amplification (default = 1.75).

Longer n-gram matches receive penalty dry_multiplier * dry_base^(match_len - dry_allowed_length).

dry_allowed_length: usize

Minimum match length (in tokens) before DRY applies any penalty (default = 2).

xtc_threshold: f32

XTC cumulative-probability threshold (0.0 = disabled; use ≥ 1.0 to disable).

The “top set” is defined as the smallest set of tokens whose cumulative probability exceeds this threshold.

xtc_probability: f32

XTC exclusion probability — how often the top-set exclusion fires (default = 0.5).

typical_p: f32

Locally-typical sampling budget (1.0 = disabled / passthrough).

Keeps only tokens whose information content is closest to the distribution entropy until cumulative probability ≥ p.

top_a: f32

Top-A adaptive threshold multiplier (0.0 = disabled).

Keeps tokens with prob >= top_a * max_prob².

eta_cutoff: f32

Eta-cutoff entropy-adaptive threshold (0.0 = disabled).

Dynamic floor = max(epsilon_cutoff, eta_cutoff / perplexity).

epsilon_cutoff: f32

Epsilon hard-floor probability used together with eta_cutoff (0.0 = no floor).

Implementations

impl SamplerConfig

pub fn greedy() -> Self

Create a greedy sampling config (always pick the most likely token).

pub fn mirostat_v2(tau: f32, eta: f32) -> Self

Create a Mirostat v2 config with the given target surprise.

Trait Implementations

impl Clone for SamplerConfig

fn clone(&self) -> SamplerConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for SamplerConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for SamplerConfig

fn default() -> Self

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for SamplerConfig

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Serialize for SamplerConfig

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.