Trait SamplingOps 

pub trait SamplingOps: Send + Sync {
    // Required methods
    fn sample_token(
        &self,
        logits: &TensorRef,
        params: &SamplingParams,
    ) -> Result<u32>;
    fn argmax(&self, logits: &TensorRef) -> Result<u32>;
}

Token sampling operations (GPU-side when possible).

Required Methods

fn sample_token( &self, logits: &TensorRef, params: &SamplingParams, ) -> Result<u32>

Sample a single token from logits using the full sampling pipeline.

logits shape: [vocab_size] or [1, vocab_size] (last-token logits).

fn argmax(&self, logits: &TensorRef) -> Result<u32>

Greedy argmax over the last dimension.

Implementors