Skip to main content

sample

oxillama_runtime::sampling

Function sample 

Source 
pub fn sample(
    logits: &[f32],
    config: &SamplerConfig,
    recent_tokens: &[u32],
) -> u32
Expand description

Sample a token ID from logits using the given configuration.

This is the stateless variant. Grammar state (if any in config) is ignored because there is no place to persist it between calls. Use Sampler for grammar-constrained generation.

§Arguments

  • logits - Raw logits from the model (length = vocab_size).
  • config - Sampling configuration.
  • recent_tokens - Recent token history for repetition penalty.

§Returns

The selected token ID.