pub struct InferenceStats {
pub feed_prompt_duration: Duration,
pub prompt_tokens: usize,
pub predict_duration: Duration,
pub predict_tokens: usize,
}
Expand description
Statistics about the inference process.
Fields§
§feed_prompt_duration: Duration
How long it took to feed the prompt.
prompt_tokens: usize
How many tokens the prompt was.
predict_duration: Duration
How long it took to predict new tokens.
predict_tokens: usize
The number of predicted tokens.
Trait Implementations§
Source§impl Clone for InferenceStats
impl Clone for InferenceStats
Source§fn clone(&self) -> InferenceStats
fn clone(&self) -> InferenceStats
Returns a duplicate of the value. Read more
1.0.0 · Source§const fn clone_from(&mut self, source: &Self)
const fn clone_from(&mut self, source: &Self)
Performs copy-assignment from
source
. Read moreSource§impl Debug for InferenceStats
impl Debug for InferenceStats
Source§impl Default for InferenceStats
impl Default for InferenceStats
Source§impl Display for InferenceStats
impl Display for InferenceStats
impl Copy for InferenceStats
Auto Trait Implementations§
impl Freeze for InferenceStats
impl RefUnwindSafe for InferenceStats
impl Send for InferenceStats
impl Sync for InferenceStats
impl Unpin for InferenceStats
impl UnwindSafe for InferenceStats
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more