Struct DecodeTrace 

pub struct DecodeTrace {
    pub prefill_ms: f64,
    pub prompt_tokens: usize,
    pub steps_ms: Vec<f64>,
}

Where the time went inside one generation.

Milliseconds, because that is the unit a reader can compare against their own patience. Per-STEP rather than an average: the first token after a prefill behaves differently from the fiftieth, and an average of the two describes neither.

Fields

prefill_ms: f64

One forward pass over the entire prompt.

prompt_tokens: usize

Prompt length in tokens — for a VLM, mostly image tokens.

steps_ms: Vec<f64>

One entry per generated token.

Implementations

impl DecodeTrace

pub fn decode_ms(&self) -> f64

Total time in the decode loop, excluding prefill.

pub fn tokens_per_sec(&self) -> f64

Generated tokens per second, prefill EXCLUDED.

Excluded because including it makes the rate depend on the size of the picture, which is not what “tokens per second” means to anyone reading it. The prefill is reported separately and in full.

Trait Implementations

impl Clone for DecodeTrace

fn clone(&self) -> DecodeTrace

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for DecodeTrace

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for DecodeTrace

fn default() -> DecodeTrace

Returns the “default value” for a type.

impl PartialEq for DecodeTrace

fn eq(&self, other: &DecodeTrace) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for DecodeTrace