Skip to main content

sample_token

sqlitegraph::inference::sampling

Function sample_token 

Source 
pub fn sample_token(logits: &[f32], temperature: f32, top_p: f32) -> usize
Expand description

Sample next token from logits using temperature + top-p (nucleus) sampling.

§Arguments

  • logits - Raw logits [vocab_size]
  • temperature - Temperature for scaling (1.0 = normal, <1 = sharper, >1 = more random)
  • top_p - Nucleus sampling threshold (0.9 = consider tokens covering 90% of probability mass)

§Returns

Token ID (index into logits array)