Struct TemperatureSampler 

pub struct TemperatureSampler {
    pub temperature: f64,
    /* private fields */
}

Samples from a softmax distribution after dividing logits by temperature.

• temperature > 1.0: flatter distribution (more randomness).
• temperature < 1.0: sharper distribution (more peaked).
• temperature = 1.0: unmodified softmax.

Fields

temperature: f64

The temperature value used to scale logits.

Implementations

impl TemperatureSampler

pub fn new(temperature: f64, seed: u64) -> Result<Self, SamplingError>

Construct a TemperatureSampler.

Returns Err(SamplingError::InvalidTemperature) if temperature <= 0.0.

pub fn sample(&mut self, logits: &[f64]) -> Result<SampledToken, SamplingError>

Sample one token from logits.

pub fn sample_batch( &mut self, logits: &[Vec<f64>], ) -> Result<Vec<SampledToken>, SamplingError>

Sample one token for each row in a batch.

Trait Implementations

impl Debug for TemperatureSampler

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.