Crate checkito

Expand description

checkito 4.0.7

A safe, efficient and simple QuickCheck-inspired library to generate shrinkable random data mainly oriented towards generative/property/exploratory testing.

§In Brief

The purpose of the library is to test general properties of a program rather than very specific examples as you would with unit tests.

When writing a checkito test (called a check), you first construct a generator by specifying the bounds that make sense for the inputs (ex: a number in the range 10..100, an alpha-numeric string, a vector of f64, etc.).
Generators can produce arbitrary complex values with their combinators, in a similar way that Iterators can.
Given a proper generator, checkito will sample the input space to find a failing case for your test.
Once a failing case is found, checkito will try to reduce the input to the simplest version of it that continues to fail (using a kind of binary search of the input space) to make the debugging process easier.
For generators with a small number of possible values (e.g., bool, a short range, or a tuple of small types), checkito will automatically run exhaustively to cover all possible values. For larger or infinite input spaces, it samples randomly. This is meant as a complement to other testing strategies.
It is recommended to write a regular unit test with the exact failing input to prevent a regression and to truly guarantee that the failing input is always tested.

§Main Concepts

The library is built around a few core traits:

Generate: is implemented for many of rust’s standard types and allows the generation of any random composite/structured data through combinator (such as tuples, Any, Map, Flatten and more). It is designed for composability and its usage should feel like working with Iterators.
Shrink: tries to reduce a generated sample to a ‘smaller’ version of it while maintaining its constraints (ex: a sample usize in the range 10..100 will never be shrunk below 10). For numbers, it means bringing the sample closer to 0, for vectors, it means removing irrelevant items and shrinking the remaining ones, and so on.
Prove: represents a desirable property of a program under test. It is used mainly in the context of the Check::check or Checker::check methods and it is the failure of a proof that triggers the shrinking process. It is implemented for a couple of standard types such as (), bool and Result. A panic!() is also considered as a failing property, thus standard assert!() macros (or any other panicking assertions) can be used to check the property.
Check: A trait (implemented for all Generate types) that provides the main entry points for running property tests: check and checks.

§Environment Variables

The behavior of the test runner can be configured through environment variables, which is particularly useful for CI environments or for debugging specific issues.

§Generation

CHECKITO_GENERATE_COUNT: Overrides the number of test cases to run. Example: CHECKITO_GENERATE_COUNT=1000 cargo test
CHECKITO_GENERATE_SIZE: Sets a fixed generation size (0.0 to 1.0). Example: CHECKITO_GENERATE_SIZE=1.0 cargo test
CHECKITO_GENERATE_SEED: Sets the initial seed for the random number generator, allowing for reproducible test runs.
CHECKITO_GENERATE_ITEMS: Sets whether to display passing generation items (true or false).

§Shrinking

CHECKITO_SHRINK_COUNT: Overrides the maximum number of shrink attempts.
CHECKITO_SHRINK_ITEMS: Sets whether to display passing shrink items (true or false).
CHECKITO_SHRINK_ERRORS: Sets whether to display failing shrink items (true or false).

§Cheat Sheet

use checkito::*;

/// The `#[check]` attribute is designed to be as thin as possible and
/// everything that is expressible with it is also ergonomically expressible as
/// _regular_ code (see below). Each `#[check]` attribute expands to a single
/// function call.
///
/// An empty `#[check]` attribute acts just like `#[test]`. It is allowed for
/// consistency between tests.
#[check]
fn empty() {}

/// The builtin `letter()` generator will yield ascii letters.
///
/// This test will be run many times with different generated values to find a
/// failing input.
#[check(letter())]
fn is_letter(value: char) {
    assert!(value.is_ascii_alphabetic());
}

/// Ranges can be used as generators and will yield values within its bounds.
///
/// A [`bool`] can be returned and if `true`, it will be considered as evidence
/// that the property under test holds.
#[check(0usize..=100)]
fn is_in_range(value: usize) -> bool {
    value <= 100
}

/// Regexes can be used and validated either dynamically using the [`regex`]
/// generator or at compile-time with the [`regex!`] macro.
///
/// Usual panicking assertions can be used in the body of the checking function
/// since a panic is considered a failed property.
#[check(regex("{", None).ok(), regex!("[a-zA-Z0-9_]*"))]
fn is_ascii(invalid: Option<String>, valid: String) {
    assert!(invalid.is_none());
    assert!(valid.is_ascii());
}

/// The `_` and `..` operators can be used to infer the [`FullGenerate`]
/// generator implementation for a type. Specifically, the `..` operator works
/// the same way as slice match patterns.
///
/// Since this test will panic, `#[should_panic]` can be used in the usual way.
#[check(..)]
#[check(_, _, _, _)]
#[check(negative::<f64>(), ..)]
#[check(.., negative::<i16>())]
#[check(_, .., _)]
#[check(negative::<f64>(), _, .., _, negative::<i16>())]
#[should_panic]
fn is_negative(first: f64, second: i8, third: isize, fourth: i16) {
    assert!(first < 0.0);
    assert!(second < 0);
    assert!(third < 0);
    assert!(fourth < 0);
}

/// `color = false` disables coloring of the output.
/// `verbose = true` will display all the steps taken by the [`check::Checker`]
/// while generating and shrinking values.
///
/// The shrinking process is pretty good at finding minimal inputs to reproduce
/// a failing property and in this case, it will always shrink values over
/// `1000` to exactly `1000`.
#[check(0u64..1_000_000, color = false, verbose = true)]
#[should_panic]
fn is_small(value: u64) {
    assert!(value < 1000);
}

/// Multiple checks can be performed.
///
/// If all generators always yield the same value, the check becomes a
/// parameterized unit test and will run only once.
#[check(3001, 6000)]
#[check(4500, 4501)]
#[check(9000, 1)]
fn sums_to_9001(left: i32, right: i32) {
    assert_eq!(left + right, 9001);
}

/// Generics can be used as inputs to the checking function.
///
/// [`Generate::map`] can be used to map a value to another.
#[check(111119)]
#[check(Generate::map(10..1000, |value| value * 10 - 1))]
#[check("a string that ends with 9")]
#[check(regex!("[a-z]*9"))]
fn ends_with_9(value: impl std::fmt::Display) -> bool {
    format!("{value}").ends_with('9')
}

pub struct Person {
    pub name: String,
    pub age: usize,
}
/// Use tuples to combine generators and build more complex structured types.
/// Alternatively implement the [`FullGenerate`] trait for the [`Person`]
/// struct.
///
/// Any generator combinator can be used here; see the other examples in the
/// _examples_ folder for more details.
///
/// Disable `debug` if a generated type does not implement [`Debug`] which
/// removes the only requirement that `#[check]` requires from input types.
#[check((letter().collect(), 18usize..=100).map(|(name, age)| Person { name, age }), debug = false)]
fn person_has_valid_name_and_is_major(person: Person) {
    assert!(person.name.is_ascii());
    assert!(person.age >= 18);
}

/// If a generator has a small domain (i.e. its cardinality is less than or
/// equal to `generate.count`), `#[check]` will automatically switch to
/// exhaustive mode and test every possible value instead of sampling randomly.
///
/// This also means that a test with only the `#[check]` attribute (no specified
/// generator) will run exactly once (since the implicit generator is `()` with
/// a cardinality of 1).
///
/// Here, `bool` has only 2 possible values and `0u8..=9` has 10, so every
/// combination (20 total) is tested exhaustively without any extra
/// configuration.
#[check(_, 0u8..=9)]
fn exhaustive_when_small_domain(sign: bool, digit: u8) {
    // Both generators are fully enumerated; no random sampling needed.
    let signed = if sign { digit as i16 } else { -(digit as i16) };
    assert!((-9..=9).contains(&signed));
}

/// Marking a checking function as `async` will automatically have its test
/// values evaluated concurrently. The concurrency level is determined by the
/// system's available parallelism.
///
/// The function body is an ordinary `async` block, so `.await` can be used
/// freely.
#[check(0u64..1000)]
async fn async_check(value: u64) {
    let doubled = async { value * 2 }.await;
    assert!(doubled < 2000);
}

/// The `#[check]` attribute essentially expands to a call to [`Check::check`]
/// with pretty printing. For some more complex scenarios, it may become more
/// convenient to simply call the [`Check::check`] manually.
///
/// The [`Generate::any`] combinator chooses from its inputs. The produced
/// `Or<..>` preserves the information about the choice but here, it can be
/// simply collapsed using [`Generate::unify<T>`].
#[test]
fn has_even_hundred() {
    (0..100, 200..300, 400..500)
        .any()
        .unify::<i32>()
        .check(|value| assert!((value / 100) % 2 == 0));
}

fn main() {
    // `checkito` comes with a bunch of builtin generators such as this generic
    // number generator. An array of generators will produce an array of values.
    let generator = &[(); 10].map(|_| number::<f64>());

    // For more configuration and control over the generation and shrinking
    // processes, retrieve a [`check::Checker`] from any generator.
    let mut checker = generator.checker();
    checker.generate.count = 1_000_000;
    checker.shrink.items = false;

    // [`check::Checker::checks`] produces an iterator of [`check::Result`] which
    // hold rich information about what happened during each check.
    for result in checker.checks(|values| values.iter().sum::<f64>() < 1000.0) {
        match result {
            check::Result::Pass(_pass) => {}
            check::Result::Shrink(_pass) => {}
            check::Result::Shrunk(_fail) => {}
            check::Result::Fail(_fail) => {}
        }
    }

    // For simply sampling random values from a generator, use [`Sample::samples`].
    // Just like in the checking process, samples will get increasingly larger.
    for _sample in generator.samples(1000) {}
}

See the examples and tests folder for more detailed examples.

§Contribute

If you find a bug or have a feature request, please open an issues.
checkito is actively maintained and pull requests are welcome.
If checkito was useful to you, please consider leaving a star!

§Alternatives

Re-exports§

pub use check::Check;
pub use generate::FullGenerate;
pub use generate::Generate;
pub use prove::Prove;
pub use sample::Sample;
pub use shrink::Shrink;

Modules§

all
any
array
boxed
cardinality
check
collect
convert
dampen
filter
filter_map
flatten
generate
keep
lazy
map
primitive
prove
regex
same
sample
shrink
size
standard
state
unify

Macros§

constant: Creates a generator from a compile-time constant value.
regex: Creates a generator from a regular expression, validated at compile time.

Functions§

any: Creates a generator that randomly chooses one of a set of generators.
array: See Generate::array.
ascii: A generator for all ASCII characters (0-127).
boxed: See Generate::boxed.
cardinality: Overrides both the static and dynamic cardinalities of a generator.
collect: See Generate::collect.
convert: See Generate::convert.
dampen: See Generate::dampen and Generate::dampen_with.
digit: A generator for ASCII digits (0-9).
filter: See Generate::filter.
filter_map: See Generate::filter_map.
flat_map: See Generate::flat_map.
flatten: See Generate::flatten.
keep: See Generate::keep.
lazy: Defers the construction of a generator until it is used.
letter: A generator for ASCII letters (a-z, A-Z).
map: See Generate::map.
negative: A generator for any non-positive Number type (includes 0).
number: A generator for the full range of any Number type.
positive: A generator for any non-negative Number type (includes 0).
regex: Creates a generator from a regular expression at runtime.
same: Creates a generator that always produces the same value.
shrinker: Creates a generator that yields the Generate::Shrink instances instead of Generate::Item.
size: See Generate::size.
unify: Unifies a generator of a orn::Or type into a single type.
with: Creates a generator from a closure that produces a value.

Attribute Macros§

check: Turns a function into a property test.

Crate checkito

Crate checkito Copy item path

checkito 4.0.7

§In Brief

§Main Concepts

§Environment Variables

§Generation

§Shrinking

§Cheat Sheet

§Contribute

§Alternatives

Re-exports§

Modules§

Macros§

Functions§

Attribute Macros§

Crate checkito