Crate dudect_bencher

This crate implements the DudeCT statistical methods for testing constant-time functions. It is based loosely off of the bencher crate.

The core idea of the DudeCT test is to construct two sets of inputs for the function in question f. The two sets of inputs will correspond to the Left distribution and the Right distribution. The user is expected to select inputs that demonstrate the potential time discrepency that they are trying to test.

For example, if f tests the equality of two given vectors, the Left distribution might contain random pairs of equal vectors, while the Right distribution might contain vectors who differ at random places or maybe in a fixed place. Once these two input sets are established, the test is performed and the statistical difference of the distribution of runtimes is calculated. If the function behaves significantly differently for inputs from Left vs Right, the runtime distributions should be significantly different. This difference will be reflected in the computed t value. See examples/ctbench-foo.rs for example code

The program output looks like

bench array_eq ... : n == +0.046M, max t = +61.61472, max tau = +0.28863, (5/tau)^2 = 300

It is interpreted as follows. Firstly note that the runtime distributions are cropped at different percentiles and about 100 t-tests are performed. Of these t-tests, the one that produces the largest absolute t-value is printed as max_t. The other values printed are

• n, indicating the number of samples used in computing this t-value
• max_tau, which is the t-value scaled for the samples size (formally, max_tau = max_t / sqrt(n))
• (5/tau)^2, which indicates the number of measurements that would be needed to distinguish the two distributions with t > 5

t-values greater than 5 are generally considered a good indication that the function is not constant time. t-values less than 5 does not necessarily imply that the function is constant-time, since there may be other input distributions under which the function behaves significantly differently.

Re-exports

pub use ctbench::BenchRng;

pub use ctbench::Class;

pub use ctbench::CtRunner;

Modules

ctbench

Macros

ctbench_main	Defines a fn main() that will run all benchmarks defined by listed functions $function. The BenchRngs given to each function are randomly seeded. Exmaple usage:
ctbench_main_with_seeds	Defines a fn main() that will run all benchmarks defined by listed functions $function and their associated seeds (if present). Seeds are represented as Option<[u32; 4]>. If None is given, a random seed will be used. The seeds are used to seed the BenchRng that's passed to each function.