Struct Sampler 

pub struct Sampler { /* private fields */ }

Token sampler.

Owns a reusable probs_buf that is grown on first use and then reused across all subsequent sample() calls, eliminating the ~1.8 MB per-call heap allocation that a fresh Vec would require for a 151 936-token vocabulary.

Implementations

impl Sampler

pub fn new(params: SamplingParams, seed: u64) -> Self

Create a new sampler with the given parameters and seed.

pub fn sample(&mut self, logits: &[f32]) -> RuntimeResult<u32>

Sample a token index from logits.

pub fn params(&self) -> &SamplingParams

Get current parameters.

pub fn set_params(&mut self, params: SamplingParams)

Replace the sampling parameters in place, preserving the PRNG state and the reusable probs_buf allocation.

Unlike constructing a fresh Sampler, this leaves rng_state untouched, so a caller can temporarily adjust (e.g.) the temperature for one request without perturbing the RNG sequence that subsequent requests on the same engine would observe.

Trait Implementations

impl Debug for Sampler

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.