Struct StandardSampler 

pub struct StandardSampler { /* private fields */ }

Selects a token after applying multiple SamplerStage’s to the probability distribution output by the model.

Implementations

impl StandardSampler

pub fn new_softmax( stages: Vec<SamplerStage>, min_keep: usize, ) -> StandardSampler

Creates a new StandardSampler that modifies the model’s raw distribution using multiple SamplerStage’s, then selects a random token from that distrubution.

Ensures that at least min_keep tokens remain after the SamplerStage’s are applied.

pub fn new_greedy() -> StandardSampler

Creates a new StandardSampler that always selects the next most token produced by the model.

pub fn new_mirostat( stages: Vec<SamplerStage>, min_keep: usize, tau: f32, eta: f32, m: i32, ) -> StandardSampler

Creates a new StandardSampler that selects a token using Mirostat.

pub fn new_mirostat_v2( stages: Vec<SamplerStage>, min_keep: usize, tau: f32, eta: f32, ) -> StandardSampler

Creates a new StandardSampler that selects a token using Mirostat V2.

Trait Implementations

impl Clone for StandardSampler

fn clone(&self) -> StandardSampler

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for StandardSampler

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for StandardSampler

fn default() -> Self

Returns the “default value” for a type.

impl Sampler for StandardSampler

fn sample( &mut self, context: *mut llama_context, tokens: &[Token], candidates_p: llama_token_data_array, ) -> Token

Given a llama_context, the tokens in context (to allow for repetition penalities), and a llama_token_data_array (which contains the distribution over the next token as output by the model), selects a token.