Skip to main content

execute_sampling

Function execute_sampling 

Source
pub fn execute_sampling(logits: &[f32], temperature: f32, top_k: usize) -> usize
Expand description

Host-side token sampling from raw logit scores.

Deterministic (greedy) when temperature == 0.0 or top_k <= 1 — returns the argmax index. Otherwise applies temperature scaling, selects the top-k candidates, runs softmax over those k values, and draws a categorical sample using an internal xorshift64 PRNG (no external crate required).