sample_with_temperature

candle_coreml::utils::sampling

Function sample_with_temperature 

Source 
pub fn sample_with_temperature(
    logits: &Tensor,
    temperature: f32,
) -> Result<i64, Error>
Expand description

Sample a token using temperature scaling

Temperature controls randomness:

  • temperature = 0.0: Greedy sampling (most likely token)
  • temperature = 1.0: Standard sampling
  • temperature > 1.0: More random
  • temperature < 1.0: More deterministic

§Arguments

  • logits - Model output logits tensor
  • temperature - Temperature for scaling

§Returns

Sampled token ID