Struct SpeculativeDecodeResult 

pub struct SpeculativeDecodeResult {
    pub generated_tokens: Vec<usize>,
    pub forward_calls: usize,
    pub tokens_generated: usize,
    pub verification_steps: usize,
    pub drafted_tokens: usize,
    pub accepted_tokens: usize,
    pub evaluated_at_position: Vec<usize>,
    pub accepted_at_position: Vec<usize>,
}

Result of a speculative decode run, with the counters that make its actual savings observable rather than just assumed.

Fields

generated_tokens: Vec<usize>

forward_calls: usize

Number of Decoder::forward_batch calls made (prefill counts as one call, each subsequent accept/reject round counts as one more, regardless of how many tokens that round produced).

tokens_generated: usize

Total tokens produced across all rounds – always equal to generated_tokens.len(), kept as a separate field so the ratio tokens_generated / forward_calls (the actual speedup metric) is easy to read directly off this struct.

verification_steps: usize

Verification rounds: forward_calls minus the prefill call. This is the denominator of the published acceptance length metric.

drafted_tokens: usize

Draft tokens the target actually evaluated. Positions past a rejection are never evaluated, so they are not counted here – counting them would deflate the accept rate by the drafter’s block size rather than by its accuracy.

accepted_tokens: usize

Draft tokens accepted.

evaluated_at_position: Vec<usize>

Per drafted position (0 = first token after the anchor), how many times that position was evaluated, i.e. reached without an earlier rejection ending the round.

accepted_at_position: Vec<usize>

Per drafted position, how many times it was accepted.

Implementations

impl SpeculativeDecodeResult

pub fn tokens_per_call(&self) -> f64

Average tokens produced per forward_batch call. 1.0 means speculation never helped (every round produced exactly the anchor token); higher means draft tokens were accepted.

pub fn acceptance_length(&self) -> Option<f64>

The published metric: completion tokens per verification step. None when nothing was verified (an empty run), because a zero there would read as “speculation made things worse” rather than “speculation did not run”.

Deliberately not the same number as Self::tokens_per_call, which charges the one-off prefill call against the average and so understates a short run.

pub fn accept_rate(&self) -> Option<f64>

Fraction of drafted positions accepted, over all positions.

pub fn accept_rate_per_position(&self) -> Vec<f64>

Accept rate at each drafted position, conditional on that position having been reached.

A single mean cannot distinguish a drafter that is uniformly mediocre from one that is excellent at position 0 and useless by position 7, and the two want opposite responses (raise the block size, or lower it). The published motivation for dFlash2’s two-tap convolution is exactly this curve falling from 99.5% to 87.8% across a block, so it has to be visible per position or the diagnosis is not testable here.

Trait Implementations

impl Clone for SpeculativeDecodeResult

fn clone(&self) -> SpeculativeDecodeResult

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for SpeculativeDecodeResult

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for SpeculativeDecodeResult

fn default() -> SpeculativeDecodeResult

Returns the “default value” for a type.