Struct PerRowSamplingArgs 

pub struct PerRowSamplingArgs<'a> {
    pub probs: TensorRef<'a, f32, 2>,
    pub top_k_arr: Option<TensorRef<'a, i32, 1>>,
    pub top_p_arr: Option<TensorRef<'a, f32, 1>>,
    pub min_p_arr: Option<TensorRef<'a, f32, 1>>,
    pub output: TensorMut<'a, i32, 1>,
    pub valid: Option<TensorMut<'a, u8, 1>>,
    pub seed_val: u64,
    pub offset_val: u64,
}

Args for a per-row sampling launch. Supply the array(s) the chosen PerRowSampler needs; leave the others None.

Fields

probs: TensorRef<'a, f32, 2>

Row-normalized probabilities [batch, vocab] f32.

top_k_arr: Option<TensorRef<'a, i32, 1>>

Per-row top-K cells [batch] i32.

top_p_arr: Option<TensorRef<'a, f32, 1>>

Per-row top-P cutoff [batch] f32.

min_p_arr: Option<TensorRef<'a, f32, 1>>

Per-row min-P multiplier [batch] f32.

output: TensorMut<'a, i32, 1>

Sampled indices [batch] i32 (written).

valid: Option<TensorMut<'a, u8, 1>>

Optional per-row “sample accepted” flags [batch] u8.

seed_val: u64

RNG seed.

offset_val: u64

RNG philox offset.