Struct SamplingConfig 

pub struct SamplingConfig {
    pub processor_chain: LogitsProcessorChain,
    pub sampler: Box<dyn Sampler>,
}

Complete sampling configuration

Fields

processor_chain: LogitsProcessorChain

sampler: Box<dyn Sampler>

Implementations

impl SamplingConfig

pub fn from_params(params: &SamplingParams) -> Self

Create from sampling parameters

pub fn supports_raw_greedy_speculation(&self) -> bool

Whether the current plan is exactly raw greedy argmax.

The legacy speculative runner only represents temperature and drafts with argmax. Any logits processor would otherwise be silently skipped.

pub fn sample( &self, ctx: SamplingContext<'_>, rng: &mut dyn RngCore, ) -> Result<TokenId>

Process logits and sample token

Trait Implementations

impl Debug for SamplingConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.