Struct DiffOut

pub struct DiffOut { /* private fields */ }

Contains the data resulting from a benchmark comparing two closures f1 and f2.

It is returned by the core benchmarking functions in this library. Its methods provide descriptive and inferential statistics about the latency samples of the two benchmarked functions, individually and in relation to each other.

All statistics involving differences refer to a value for f1 minus the corresponding value for f2.

Implementations

impl DiffOut

pub fn n(&self) -> u64

Number of observations (sample size) for a function, as an integer.

It is the same value for f1 and f2.

pub fn nf(&self) -> f64

Number of observations (sample size) for a function, as a floating point number.

It is the same value for f1 and f2.

pub fn summary_f1(&self) -> SummaryStats

Summary descriptive statistics for f1.

Includes sample size, mean, standard deviation, median, several percentiles, min, and max.

pub fn summary_f2(&self) -> SummaryStats

Summary descriptive statistics for f2.

Includes sample size, mean, standard deviation, median, several percentiles, min, and max.

pub fn mean_f1(&self) -> f64

Mean of f1’s latencies.

pub fn mean_f2(&self) -> f64

Mean of f1’s latencies.

pub fn median_f1(&self) -> f64

Median of f1’s latencies.

pub fn median_f2(&self) -> f64

Median of f2’s latencies.

pub fn diff_medians_f1_f2(&self) -> f64

Difference between the median of f1’s latencies and the median of f2’s latencies.

pub fn ratio_medians_f1_f2(&self) -> f64

Ratio of the median of f1’s latencies to the median of f2’s latencies.

pub fn count_f1_lt_f2(&self) -> u64

Count of paired observations where f1’s latency is less than f2’s.

pub fn count_f1_eq_f2(&self) -> u64

Count of paired observations where f1’s latency is equal to f2’s.

pub fn count_f1_gt_f2(&self) -> u64

Count of paired observations where f1’s latency is greater than f2’s.

pub fn mean_ln_f1(&self) -> f64

Mean of the natural logarithms of f1’s latencies.

pub fn stdev_ln_f1(&self) -> f64

Standard deviation of the natural logarithms f1’s latecies.

pub fn mean_ln_f2(&self) -> f64

Mean of the natural logarithms of f2’s latencies.

pub fn stdev_ln_f2(&self) -> f64

Standard deviation of the natural logarithms f2’s latecies.

pub fn mean_diff_f1_f2(&self) -> f64

Mean of the differences between paired latencies of f1 and f2. Equal to the difference between the mean of f1’s latencies and the mean of f2’s latencies.

pub fn stdev_diff_f1_f2(&self) -> f64

Standard deviation of the differences between paired latencies of f1 and f2. (Not the difference between the standard deviation of f1’s latencies and the standard deviation off2’s latencies.)

pub fn mean_diff_ln_f1_f2(&self) -> f64

Mean of the differences between the natural logarithms of paired latencies of f1 and f2. (Same as the difference between the mean of the natural logarithms of f1’s latencies and the mean of the natural logarithms off2’s latencies.)

pub fn stdev_diff_ln_f1_f2(&self) -> f64

Standard deviation of the differences between the natural logarithms of paired latencies of f1 and f2. (Not the difference between the standard deviation of the natural logarithms of f1’s latencies and the standard deviation of the natural logarithms off2’s latencies.)

pub fn ratio_medians_f1_f2_from_lns(&self) -> f64

Estimated ratio of the median f1 latency to the median f2 latency, computed as the exp() of Self::mean_diff_ln_f1_f2.

pub fn welch_ln_t(&self) -> f64

Welch’s t statistic for mean(ln(latency(f1))) - mean(ln(latency(f2))) (where ln is the natural logarithm).

pub fn welch_ln_df(&self) -> f64

Degrees of freedom for Welch’s t-test for mean(ln(latency(f1))) - mean(ln(latency(f2))) (where ln is the natural logarithm).

pub fn welch_ln_ci(&self, alpha: f64) -> Ci

Welch confidence interval for mean(ln(latency(f1))) - mean(ln(latency(f2))) (where ln is the natural logarithm), with confidence level (1 - alpha).

Assumes that both latency(f1) and latency(f2) are approximately log-normal. This assumption is widely supported by performance analysis theory and empirical data.

This is also the confidence interval for the difference of medians of logarithms under the above assumption.

pub fn welch_ratio_ci(&self, alpha: f64) -> Ci

Welch confidence interval for median(latency(f1)) / median(latency(f2)), with confidence level (1 - alpha).

Assumes that both latency(f1) and latency(f2) are approximately log-normal. This assumption is widely supported by performance analysis theory and empirical data.

pub fn welch_value_position_wrt_ratio_ci( &self, value: f64, alpha: f64, ) -> PositionWrtCi

Position of value with respect to the Welch confidence interval for median(latency(f1)) / median(latency(f2)), with confidence level (1 - alpha).

Assumes that both latency(f1) and latency(f2) are approximately log-normal. This assumption is widely supported by performance analysis theory and empirical data.

pub fn welch_ln_test(&self, alt_hyp: AltHyp, alpha: f64) -> HypTestResult

Welch’s test of the hypothesis that median(latency(f1)) == median(latency(f2)), with alternative hypothesis alt_hyp and confidence level (1 - alpha).

Assumes that both latency(f1) and latency(f2) are approximately log-normal. This assumption is widely supported by performance analysis theory and empirical data.

pub fn student_diff_ln_t(&self) -> f64

👎Deprecated: Use welch_ln_t instead

Student’s one-sample t statistic for mean(ln(latency(f1)) - ln(latency(f2))) (where ln is the natural logarithm).

pub fn student_diff_ln_df(&self) -> f64

👎Deprecated: Use welch_ln_df instead

Degrees of freedom for Student’s one-sample t-test for mean(ln(latency(f1)) - ln(latency(f2))) (where ln is the natural logarithm).

pub fn student_diff_ln_ci(&self, alpha: f64) -> Ci

👎Deprecated: Use welch_ln_ci instead

Student’s one-sample confidence interval for mean(ln(latency(f1)) - ln(latency(f2))) (where ln is the natural logarithm). with confidence level (1 - alpha).

Assumes that both latency(f1) and latency(f2) are approximately log-normal. This assumption is widely supported by performance analysis theory and empirical data.

pub fn student_ratio_ci(&self, alpha: f64) -> Ci

👎Deprecated: Use welch_ratio_ci instead

Student’s one-sample confidence interval for median(latency(f1)) / median(latency(f2)), with confidence level (1 - alpha).

Assumes that both latency(f1) and latency(f2) are approximately log-normal. This assumption is widely supported by performance analysis theory and empirical data.

pub fn student_value_position_wrt_ratio_ci( &self, value: f64, alpha: f64, ) -> PositionWrtCi

👎Deprecated: Use welch_value_position_wrt_ratio_ci instead

Position of value with respect to Student’s one-sample confidence interval for median(latency(f1)) / median(latency(f2)), with confidence level (1 - alpha).

Assumes that both latency(f1) and latency(f2) are approximately log-normal. This assumption is widely supported by performance analysis theory and empirical data.

pub fn student_diff_ln_test(&self, alt_hyp: AltHyp, alpha: f64) -> HypTestResult

👎Deprecated: Use welch_ln_test instead

Student’s one-sample test of the hypothesis that median(latency(f1)) == median(latency(f2)), with alternative hypothesis alt_hyp and confidence level (1 - alpha).

Assumes that both latency(f1) and latency(f2) are approximately log-normal. This assumption is widely supported by performance analysis theory and empirical data.