[−][src]Crate dudect_bencher

This crate implements the DudeCT statistical methods for testing constant-time functions. It is based loosely off of the `bencher` crate.

The core idea of the DudeCT test is to construct two sets of inputs for the function in question `f`. The two sets of inputs will correspond to the `Left` distribution and the `Right` distribution. The user is expected to select inputs that demonstrate the potential time discrepency that they are trying to test.

For example, if `f` tests the equality of two given vectors, the `Left` distribution might contain random pairs of equal vectors, while the `Right` distribution might contain vectors who differ at random places or maybe in a fixed place. Once these two input sets are established, the test is performed and the statistical difference of the distribution of runtimes is calculated. If the function behaves significantly differently for inputs from `Left` vs `Right`, the runtime distributions should be significantly different. This difference will be reflected in the computed t value. See `examples/ctbench-foo.rs` for example code

The program output looks like

``````bench array_eq ... : n == +0.046M, max t = +61.61472, max tau = +0.28863, (5/tau)^2 = 300
``````

It is interpreted as follows. Firstly note that the runtime distributions are cropped at different percentiles and about 100 t-tests are performed. Of these t-tests, the one that produces the largest absolute t-value is printed as `max_t`. The other values printed are

• `n`, indicating the number of samples used in computing this t-value
• `max_tau`, which is the t-value scaled for the samples size (formally, `max_tau = max_t / sqrt(n)`)
• `(5/tau)^2`, which indicates the number of measurements that would be needed to distinguish the two distributions with t > 5

t-values greater than 5 are generally considered a good indication that the function is not constant time. t-values less than 5 does not necessarily imply that the function is constant-time, since there may be other input distributions under which the function behaves significantly differently.

Re-exports

 `pub use ctbench::BenchRng;` `pub use ctbench::Class;` `pub use ctbench::CtRunner;`

Modules

 ctbench

Macros

 ctbench_main Defines a `fn main()` that will run all benchmarks defined by listed functions `\$function`. The `BenchRng`s given to each function are randomly seeded. Exmaple usage: ctbench_main_with_seeds Defines a `fn main()` that will run all benchmarks defined by listed functions `\$function` and their associated seeds (if present). Seeds are represented as `Option<[u32; 4]>`. If `None` is given, a random seed will be used. The seeds are used to seed the `BenchRng` that's passed to each function.