Struct SpeculativeSamplingDescriptor 

pub struct SpeculativeSamplingDescriptor {
    pub batch_size: i32,
    pub num_speculative_tokens: i32,
    pub vocab_size: i32,
    pub deterministic: bool,
}

Descriptor for a speculative-decode verification op.

Fields

batch_size: i32

Number of requests.

num_speculative_tokens: i32

Draft tokens proposed per request.

vocab_size: i32

Vocabulary size.

deterministic: bool

Sort-based tiebreak on ambiguous cells.

Trait Implementations

impl Clone for SpeculativeSamplingDescriptor

fn clone(&self) -> SpeculativeSamplingDescriptor

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for SpeculativeSamplingDescriptor

impl Debug for SpeculativeSamplingDescriptor

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.