Struct PerfContextData 

pub struct PerfContextData { /* private fields */ }

A wrapper around llama_perf_context_data.

Implementations

impl PerfContextData

pub fn new( t_start_ms: f64, t_load_ms: f64, t_p_eval_ms: f64, t_eval_ms: f64, n_p_eval: i32, n_eval: i32, ) -> Self

Create a new PerfContextData.

let timings = PerfContextData::new(1.0, 2.0, 3.0, 4.0, 5, 6);
assert_eq!(timings.t_load_ms(), 2.0);
assert_eq!(timings.t_p_eval_ms(), 3.0);
assert_eq!(timings.t_eval_ms(), 4.0);
assert_eq!(timings.n_p_eval(), 5);
assert_eq!(timings.n_eval(), 6);

pub fn print(ctx: &LlamaContext<'_>)

print llama context performance data load time prompt eval time eval time total time

pub fn t_start_ms(&self) -> f64

Get the start time in milliseconds.

pub fn t_end_ms(&self) -> f64

Get the end time in milliseconds.

pub fn t_load_ms(&self) -> f64

Get the load time in milliseconds.

pub fn t_p_eval_ms(&self) -> f64

Get the prompt evaluation time in milliseconds.

pub fn t_eval_ms(&self) -> f64

Get the evaluation time in milliseconds.

pub fn n_p_eval(&self) -> i32

Get the number of prompt evaluations.

pub fn n_eval(&self) -> i32

Get the number of evaluations.

pub fn n_reused(&self) -> i32

How many times llama.cpp reused a cached compute graph instead of rebuilding it.

Graph reuse is the fast path. A low count against a high Self::n_eval means something is invalidating the graph every step — a changing batch shape, or toggling a context flag mid-generation.

pub fn n_reallocs_same(&self) -> i32

Graph reallocations where the graph size was unchanged.

These should be rare: the shape did not change, so the buffers should have been reusable. A climbing count points at allocator churn rather than at your batching.

pub fn n_reallocs_diff(&self) -> i32

Graph reallocations where the graph size changed.

Expected whenever the batch shape moves — a prefill of a different length, or switching between prompt and single-token decode.

pub fn set_t_start_ms(&mut self, t_start_ms: f64)

Set the start time in milliseconds.

pub fn set_t_load_ms(&mut self, t_load_ms: f64)

Set the load time in milliseconds.

pub fn set_t_p_eval_ms(&mut self, t_p_eval_ms: f64)

Set the prompt evaluation time in milliseconds.

pub fn set_t_eval_ms(&mut self, t_eval_ms: f64)

Set the evaluation time in milliseconds.

pub fn set_n_p_eval(&mut self, n_p_eval: i32)

Set the number of prompt evaluations.

pub fn set_n_eval(&mut self, n_eval: i32)

Set the number of evaluations.

Trait Implementations

impl Clone for PerfContextData

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for PerfContextData

impl Debug for PerfContextData

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Display for PerfContextData

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.