Struct SamplingKernel 

pub struct SamplingKernel { /* private fields */ }

GPU sampling kernel — owns compiled pipelines for softmax, top-k, and categorical sampling.

Construct with SamplingKernel::new. All heavy GPU resources (pipelines, bind-group layouts) are created once at construction time and reused across calls.

Implementations

impl SamplingKernel

pub fn new(_context: ()) -> GpuResult<Self>

Stub constructor when the gpu feature is disabled.

Always returns Err(GpuError::NoAdapter).

pub fn softmax(&self, logits: &[f32], temperature: f32) -> GpuResult<Vec<f32>>

Apply temperature scaling and compute softmax probabilities.

• logits — raw logit vector (host slice), length n_vocab.
• temperature — sampling temperature. 0.0 → argmax (degenerate distribution with 1.0 at the argmax, 0.0 elsewhere).

Returns a host Vec<f32> of normalised probabilities.

pub fn top_k(&self, probs: &[f32], k: usize) -> GpuResult<(Vec<f32>, Vec<u32>)>

Extract top-k probability/index pairs.

• probs — normalised probability distribution, host slice of length n_vocab.
• k — number of candidates to extract. Must satisfy k ≤ n_vocab.

Returns (topk_probs, topk_idxs) as host Vecs of length k.

pub fn sample(&self, probs: &[f32], idxs: &[u32], seed: u64) -> GpuResult<u32>

Sample one token from a probability distribution.

• probs — probability values (need not sum to 1.0; the shader walks the raw CDF, so partial sums work too as long as the uniform variate is within range).
• idxs — token IDs corresponding to each entry in probs.
• seed — 64-bit seed for the LCG RNG.

Returns the sampled token ID as a u32.