Struct PerfMetrics 

pub struct PerfMetrics {
    pub t_load_ms: u64,
    pub t_p_eval_ms: u64,
    pub t_eval_ms: u64,
    pub n_p_eval: u32,
    pub n_eval: u32,
    pub n_samples: u32,
}

Performance metrics breakdown for inference phases.

Tracks timing for each phase of LLM inference:

• Model loading (t_load_ms)
• Prompt evaluation / prefill (t_p_eval_ms)
• Token generation / decode (t_eval_ms)

Example

use trueno::brick::PerfMetrics;

let mut metrics = PerfMetrics::default();
metrics.record_load(1500);  // 1.5s model load
metrics.record_prefill(200, 512);  // 200ms for 512 prompt tokens
metrics.record_decode(50);  // 50ms per generated token

println!("{}", metrics.summary());

Fields

t_load_ms: u64

Model loading time (milliseconds)

t_p_eval_ms: u64

Prompt evaluation time - prefill phase (milliseconds)

t_eval_ms: u64

Token generation time - decode phase (milliseconds)

n_p_eval: u32

Number of tokens in prompt (prefill)

n_eval: u32

Number of tokens generated (decode)

n_samples: u32

Sample count for t_eval (for averaging)

Implementations

impl PerfMetrics

pub fn new() -> Self

Create new metrics instance.

pub fn record_load(&mut self, ms: u64)

Record model loading time.

pub fn record_prefill(&mut self, ms: u64, tokens: u32)

Record prefill (prompt evaluation) time.

pub fn record_decode(&mut self, ms: u64)

Record a single decode step.

pub fn record_decode_batch(&mut self, ms: u64, tokens: u32)

Record batch decode step.

pub fn tokens_per_second(&self) -> f64

Tokens per second during generation (decode throughput).

pub fn prefill_tokens_per_second(&self) -> f64

Tokens per second during prompt evaluation (prefill throughput).

pub fn total_ms(&self) -> u64

Total time for complete inference.

pub fn time_to_first_token_ms(&self) -> u64

Time-to-first-token (TTFT).

pub fn avg_token_latency_ms(&self) -> f64

Average time per token during decode.

pub fn summary(&self) -> String

Formatted summary string.

pub fn reset(&mut self)

Reset all metrics.

Trait Implementations

impl Clone for PerfMetrics

fn clone(&self) -> PerfMetrics

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for PerfMetrics

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for PerfMetrics

fn default() -> PerfMetrics

Returns the “default value” for a type.