Struct SpeculativeDecodeResult 

pub struct SpeculativeDecodeResult {
    pub generated_tokens: Vec<usize>,
    pub forward_calls: usize,
    pub tokens_generated: usize,
}

Result of a speculative decode run, with the counters that make its actual savings observable rather than just assumed.

Fields

generated_tokens: Vec<usize>

forward_calls: usize

Number of Decoder::forward_batch calls made (prefill counts as one call, each subsequent accept/reject round counts as one more, regardless of how many tokens that round produced).

tokens_generated: usize

Total tokens produced across all rounds – always equal to generated_tokens.len(), kept as a separate field so the ratio tokens_generated / forward_calls (the actual speedup metric) is easy to read directly off this struct.

Implementations

impl SpeculativeDecodeResult

pub fn tokens_per_call(&self) -> f64

Average tokens produced per forward_batch call. 1.0 means speculation never helped (every round produced exactly the anchor token); higher means draft tokens were accepted.

Trait Implementations

impl Clone for SpeculativeDecodeResult

fn clone(&self) -> SpeculativeDecodeResult

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for SpeculativeDecodeResult

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.