dicetest

Framework for writing tests with randomly generated test data.

For more information see the guide.

Features

These features are available:

Generators for many libstd types (u8, String, Vec, etc.).
Generators for functions (FnMut, FnOnce, Fn).
Generator combinators (map, flat_map, zip, etc.).
Derivable trait Dice for implementing generators.
Configurable test runner Dicetest.
Utilities for debugging tests (hints and stats).

These features are missing:

Shrinking of counterexamples.
Custom pseudorandom number generators.

Example

Here's an example of an incorrect sort function tested with dicetest:

fn bubble_sort<T: Ord>(slice: &mut [T]) {
    let len = slice.len();

    for _ in 0..len {
        for j in 1..len - 1 {
            let jpp = j + 1;
            if slice[j] > slice[jpp] {
                slice.swap(j, jpp);
            }
        }
    }
}

#[cfg(test)]
mod tests {
    use super::*;
    use dicetest::prelude::*;

    #[test]
    fn result_of_bubble_sort_is_sorted() {
        Dicetest::repeatedly().run(|mut fate| {
            let mut vec: Vec<u8> = fate.roll(die());
            hint!("unsorted: {:?}", vec);

            bubble_sort(&mut vec);
            hint!("  sorted: {:?}", vec);

            let is_sorted = vec.windows(2).all(|w| w[0] <= w[1]);
            assert!(is_sorted);
        })
    }
}

Running cargo test produces the following output:

The test failed after 9 passes.

# Config
- seed: 6411673118948708013
- start limit: 0
- end limit: 100
- passes: 200

# Counterexample
- run code: 3lTBtDxQx6SneW3r4sNLUVoYAREJ8OuO9B0yp31nna0NdwFGFvA4no
- limit: 4
- hints:
        - unsorted: [54, 164, 2]
        -   sorted: [54, 2, 164]
- error: assertion failed: is_sorted

You can rerun the counterexample by setting an environment variable:

DICETEST_DEBUG=3lTBtDxQx6SneW3r4sNLUVoYAREJ8OuO9B0yp31nna0NdwFGFvA4no cargo test

Or you can modify the test:

Dicetest::debug("3lTBtDxQx6SneW3r4sNLUVoYAREJ8OuO9B0yp31nna0NdwFGFvA4no").run(|mut fate| {
    // ...
})

After fixing the bug you can keep the counterexample as a regression test:

Dicetest::repeatedly()
    .regression("3lTBtDxQx6SneW3r4sNLUVoYAREJ8OuO9B0yp31nna0NdwFGFvA4no")
    .run(|mut fate| {
        // ...
    })

Alternatives

Write down your test data and use a loop.
Use the crate arbitrary and arbtest.
Use the crate quickcheck.
Use the crate proptest.

License

Licensed under either of

Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

at your option.

Contribution