Struct Multinomial 

pub struct Multinomial { /* private fields */ }

A Sampler which chooses a token ID according to the probability of each logit.

Input logits are first normalized using a softmax operation before a token ID is sampled according to the probability of each logit.

By default sampling uses a random seed so results will vary for each run. To get repeatable sampling, use with_seed.

Implementations

impl Multinomial

pub fn new() -> Self

Create a sampler with a random seed.

pub fn with_seed(seed: u64) -> Self

Create a sampler with a fixed seed.

This guarantees repeatable sampling.

Trait Implementations

impl Clone for Multinomial

fn clone(&self) -> Multinomial

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Default for Multinomial

fn default() -> Multinomial

Returns the “default value” for a type.

impl Sampler for Multinomial

fn sample(&self, logits: &Logits) -> TokenId

Sample a token ID from the output logits of a model.