Crate test_that

Expand description

A rich test assertion library for Rust.

This crate provides:

A framework for writing matchers which can be combined to make a wide range of assertions on data,
A rich set of matchers, and
A set of test assertion macros.

§Assertions and matchers

The core of Test That! is its matchers. Matchers indicate what aspect of an actual value one is asserting: (in-)equality, containment, regular expression matching, and so on.

To make an assertion using a matcher, Test That! offers three macros:

assert_that! panics if the assertion fails, aborting the test.
expect_that! logs an assertion failure, marking the test as having failed, but allows the test to continue running (called a non-fatal assertion). It requires the use of the test_that::test attribute macro on the test itself.
verify_that! has no side effects and evaluates to a Result whose Err variant describes the assertion failure, if there is one. In combination with the ? operator, this can be used to abort the test on assertion failure without panicking. It is also the building block for the other two macros above.

For example:

use test_that::prelude::*;

#[test]
fn fails_and_panics() {
    let value = 2;
    assert_that!(value, eq(4));
}

#[test_that::test]
fn two_logged_failures() {
    let value = 2;
    expect_that!(value, eq(4)); // Test now failed, but continues executing.
    expect_that!(value, eq(5)); // Second failure is also logged.
}

#[test]
fn fails_immediately_without_panic() -> TestResult<()> {
    let value = 2;
    verify_that!(value, eq(4))?; // Test fails and aborts.
    verify_that!(value, eq(2))?; // Never executes.
    Ok(())
}

#[test]
fn simple_assertion() -> TestResult<()> {
    let value = 2;
    verify_that!(value, eq(4)) // One can also just return the last assertion.
}

Matchers are composable:

use test_that::prelude::*;

#[test_that::test]
fn contains_at_least_one_item_at_least_3() {
    let value = vec![1, 2, 3];
    expect_that!(value, contains(ge(3)));
}

They can also be logically combined:

use test_that::prelude::*;

#[test_that::test]
fn strictly_between_9_and_11() {
    let value = 10;
    expect_that!(value, gt(9).and(not(ge(11))));
}
#[test_that::test]
fn outside_interval_from_0_to_2() {
    let value = 10;
    expect_that!(value, lt(0).or(gt(2)));
}

§Available matchers

The following matchers are provided in Test That!:

Matcher	What it matches
`all!`	Anything matched by all given matchers.
`any!`	Anything matched by at least one of the given matchers.
`anything`	Any input.
`approx_eq`	A floating point number within a standard tolerance of the argument.
`char_count`	A string with a Unicode scalar count matching the argument.
`container_eq`	Same as `eq`, but for containers (with a better mismatch description).
`contains`	A container containing an element matched by the given matcher.
`contains_each!`	A container containing distinct elements each of the arguments match.
`contains_exactly!`	A container whose elements the arguments match, in any order.
`contains_regex`	A string containing a substring matching the given regular expression.
`contains_substring`	A string containing the given substring.
`displays_as`	A `Display` value whose formatted string is matched by the argument.
`each`	A container all of whose elements the given argument matches.
`empty`	An empty collection.
`ends_with`	A string ending with the given suffix.
`eq`	A value equal to the argument, in the sense of the `PartialEq` trait.
`eq_deref_of`	A value equal to the dereferenced value of the argument.
`err`	A `Result` containing an `Err` variant the argument matches.
`field!`	A struct or enum with a given field whose value the argument matches.
`ge`	A `PartialOrd` value greater than or equal to the given value.
`gt`	A `PartialOrd` value strictly greater than the given value.
`has_entry`	A `HashMap` containing a given key whose value the argument matches.
`in_range`	An item (of an orderable type) contained in the given range.
`is_contained_in!`	A container each of whose elements is matched by some given matcher.
`is_finite`	A floating point number which is finite.
`is_infinite`	A floating point number which is infinite.
`is_nan`	A floating point number which is NaN.
`le`	A `PartialOrd` value less than or equal to the given value.
`len`	A container whose number of elements the argument matches.
`lt`	A `PartialOrd` value strictly less than the given value.
`matches_pattern!`	A struct or enum whose fields are matched according to the arguments.
`matches_regex`	A string matched by the given regular expression.
`near`	A floating point number within a given tolerance of the argument.
`none`	An `Option` containing `None`.
`not`	Any value the argument does not match.
`ok`	A `Result` containing an `Ok` variant the argument matches.
`pat!`	Alias for `matches_pattern!`.
`points_to`	Any `Deref` such as `&`, `Rc`, etc. whose value the argument matches.
`pointwise!`	A container whose contents the arguments match in a pointwise fashion.
`predicate`	A value on which the given predicate returns true.
`result_of!`	The result of applying the given closure matched by the given matcher.
`some`	An `Option` containing `Some` whose value the argument matches.
`starts_with`	A string starting with the given prefix.
`subset_of`	A container all of whose elements are contained in the argument.
`superset_of`	A container containing all elements of the argument.

§Matching on complex data structures

One can compose matcher invocations to express conditions on complex data structures.

#[derive(Debug)]
struct StructWithVec {
    vec: Vec<String>,
}
let value = StructWithVec { vec: vec!["Hello, world!".into()] };
assert_that!(value, matches_pattern!(StructWithVec {
    vec: contains(starts_with("Hello")),
}));

The matchers follow a parallel structure to the data structure being matched. In general, owned values are matched against owned values, references against references, and so on. When matching against a reference, one can use points_to to “dereference” it.

#[derive(Debug)]
struct StructWithRef<'a> {
    reference: &'a u32
}
let inner = 1234;
let value = StructWithRef { reference: &inner };
assert_that!(value, matches_pattern!(StructWithRef {
    reference: points_to(gt(1000)),
}));

The assertion macros as well as the matcher matches_pattern! (and its alias pat!) support a shorthand notation for this using the * sigil:

#[derive(Debug)]
struct StructWithRef<'a> {
    reference: &'a u32
}
let inner = 1234;
let value = StructWithRef { reference: &inner };
assert_that!(value, matches_pattern!(StructWithRef {
    *reference: gt(1000),
}));

One does not dereference string slices when matching against them.

#[derive(Debug)]
struct StructWithVec<'a> {
    vec: Vec<&'a str>,
}
let value = StructWithVec { vec: vec!["Hello, world!"] };
assert_that!(value, matches_pattern!(StructWithVec {
    vec: contains(starts_with("Hello")),
}));

One does, however, dereference array slices.

#[derive(Debug)]
struct StructWithSlice<'a> {
    slice: &'a [u32],
}
let inner = [1, 2, 3];
let value = StructWithSlice { slice: &inner };
assert_that!(value, matches_pattern!(StructWithSlice {
    *slice: contains(eq(2)),
}));

Both points_to and the * sigil also work with smart pointers such as Box, Rc and Arc.

#[derive(Debug)]
struct StructWithBox {
    boxed: Box<u32>
}
let value = StructWithBox { boxed: Box::new(1234) };
assert_that!(value, matches_pattern!(StructWithBox {
    boxed: points_to(gt(1000)),
}));
assert_that!(value, matches_pattern!(StructWithBox {
    *boxed: gt(1000),
}));

§Matching on tuples

One can match on a plain tuple of items by constructing a tuple of matchers.

let value = (1, "Hello, world");
assert_that!(value, (eq(1), ends_with("world")));

All fields must be covered by matchers. Use anything for fields which are not relevant for the test.

verify_that!((123, 456), (eq(123), anything()))

This supports tuples of up to 12 elements. Tuples longer than that do not automatically inherit the Debug trait from their members, so are generally not supported; see Rust by Example.

§Writing matchers

One can extend the library by writing additional matchers. To do so, create a struct holding the matcher’s data and have it implement the traits Matcher and Describable:

use test_that::{description::Description, matcher::{Describable, Matcher, MatcherResult}};
use std::fmt::Debug;

pub struct MyEqMatcher<T> {
    expected: T,
}

impl<T: PartialEq + Debug> Matcher<T> for MyEqMatcher<T> {
    fn matches(&self, actual: &T) -> MatcherResult {
        if self.expected == *actual {
            MatcherResult::Match
        } else {
            MatcherResult::NoMatch
        }
    }
}

impl<T: Debug> Describable for MyEqMatcher<T> {
    fn describe(&self, matcher_result: MatcherResult) -> Description {
        match matcher_result {
            MatcherResult::Match => {
                format!("is equal to {:?} the way I define it", self.expected).into()
            }
            MatcherResult::NoMatch => {
                format!("isn't equal to {:?} the way I define it", self.expected).into()
            }
        }
    }
}

One should also expose a function which constructs the matcher:

pub fn eq_my_way<T>(expected: T) -> MyEqMatcher<T> {
    MyEqMatcher { expected }
}

The new matcher can then be used in the assertion macros:

#[test_that::test]
fn should_be_equal_by_my_definition() {
    expect_that!(10, eq_my_way(10));
}

§Non-fatal assertions

Using non-fatal assertions, a single test is able to log multiple assertion failures. Any single assertion failure causes the test to be considered having failed, but execution continues until the test completes or otherwise aborts.

To make a non-fatal assertion, use the macro expect_that!. The test must also be marked with test_that::test instead of the Rust-standard #[test].

use test_that::prelude::*;

#[test_that::test]
fn three_non_fatal_assertions() {
    let value = 2;
    expect_that!(value, eq(2));  // Passes; test still considered passing.
    expect_that!(value, eq(3));  // Fails; logs failure and marks the test failed.
    expect_that!(value, eq(4));  // A second failure, also logged.
}

This can be used in the same tests as verify_that!, in which case the test function must also return TestResult<()>:

use test_that::prelude::*;

#[test_that::test]
fn failing_non_fatal_assertion() -> TestResult<()> {
    let value = 2;
    expect_that!(value, eq(3));  // Just marks the test as having failed.
    verify_that!(value, eq(2))?;  // Passes, so does not abort the test.
    Ok(())        // Because of the failing expect_that! call above, the
                  // test fails despite returning Ok(())
}

use test_that::prelude::*;

#[test_that::test]
fn failing_fatal_assertion_after_non_fatal_assertion() -> TestResult<()> {
    let value = 2;
    expect_that!(value, eq(2));  // Passes; test still considered passing.
    verify_that!(value, eq(3))?; // Fails and aborts the test.
    expect_that!(value, eq(3));  // Never executes, since the test already aborted.
    Ok(())
}

§Predicate assertions

The macro verify_pred! provides predicate assertions analogous to GoogleTest’s EXPECT_PRED family of macros. Wrap an invocation of a predicate in a verify_pred! invocation to turn that into a test assertion which passes precisely when the predicate returns true:

fn stuff_is_correct(x: i32, y: i32) -> bool {
    x == y
}

let x = 3;
let y = 4;
verify_pred!(stuff_is_correct(x, y))?;

The assertion failure message shows the arguments and the values to which they evaluate:

stuff_is_correct(x, y) was false with
  x = 3,
  y = 4

The verify_pred! invocation evaluates to a TestResult<()> just like verify_that!. There is also a macro expect_pred! to make a non-fatal predicate assertion.

§Unconditionally generating a test failure

The macro fail! unconditionally evaluates to a Result indicating a test failure. It can be used analogously to verify_that! and verify_pred! to cause a test to fail, with an optional formatted message:

#[test]
fn always_fails() -> TestResult<()> {
    fail!("This test must fail with {}", "today")
}

§Integrations with other crates

§Non-fatal assertions with other test libraries

Test That! requires the use of the test attribute macro to enable non-fatal assertions. This integrates with other common test attribute macros such as tokio::test and rstest. Just apply both attribute macros to your test.

Note: In the case of rstest, make sure to put #[test_that::test] before #[rstest]. Otherwise the annotated test will run twice, since both macros will attempt to register a test with the Rust test harness.

§Converting errors into test failures

Test That! includes integrations with the Anyhow and Proptest crates to simplify turning errors from those crates into test failures.

To use this, activate the anyhow, respectively proptest feature in Test That! and invoke the extension method or_fail() on a Result value in your test. For example:

#[test]
fn has_anyhow_failure() -> TestResult<()> {
    Ok(just_return_error().or_fail()?)
}

fn just_return_error() -> anyhow::Result<()> {
    anyhow::Result::Err(anyhow!("This is an error"))
}

One can convert Proptest test failures into Test That! test failures when the test is invoked with TestRunner::run:

#[test]
fn numbers_are_greater_than_zero() -> TestResult<()> {
    let mut runner = TestRunner::new(Config::default());
    runner.run(&(1..100i32), |v| Ok(verify_that!(v, gt(0))?)).or_fail()
}

Similarly, when the proptest feature is enabled, Test That! assertion failures can automatically be converted into Proptest TestCaseError through the ? operator as the example above shows.

§Features and dependencies

All dependencies of Test That! are optional. The default features include:

Support for non-fatal assertions,
Support for matching against regular expressions,
Support for matching floating point numbers, and
Features requiring std.

Test That! also runs in nostd environments. It requires an allocator.

§Note on API stability

This crate generally follows semantic versioning conventions. However, there are several symbols which must be part of the public API surface for technical reasons, but which downstream crates are not intended to use directly. It may be necessary to make incompatible changes to those APIs to fix defects in the future.

All such symbols are placed in submodules named __internal. Please do not use such symbols directly but instead stick to the official API surface.

Re-exports§

pub use result::OrFailExt;
pub use result::TestResult;
pub use result::TestResultExt;
pub use result::verify_current_test_outcome;

Modules§

compatgoogletest-compat: Aliases to ease porting from GoogleTest Rust.
description
matcher: The components required to implement matchers.
matcher_support: Utilities to facilitate writing matchers.
matchers: All built-in matchers of this crate are in submodules of this module.
prelude: Re-exports of the symbols in this crate which are most likely to be used.
result

Macros§

assert_pred: Asserts that the given predicate applied to the given arguments returns true, panicking if it does not.
assert_that: Matches the given value against the given matcher, panicking if it does not match.
expect_prednon-fatal-assertions: Asserts that the given predicate applied to the given arguments returns true, failing the test but continuing execution if not.
expect_thatnon-fatal-assertions: Matches the given value against the given matcher, marking the test as failed but continuing execution if it does not match.
fail: Evaluates to a Result which contains an Err variant with the given test failure message.
verify_pred: Asserts that the given predicate applied to the given arguments returns true.
verify_that: Checks whether the Matcher given by the second argument matches the first argument.

Attribute Macros§

testnon-fatal-assertions: Marks a test which may have non fatal assertions.

Crate test_that

Crate test_that Copy item path

§Assertions and matchers

§Available matchers

§Matching on complex data structures

§Matching on tuples

§Writing matchers

§Non-fatal assertions

§Predicate assertions

§Unconditionally generating a test failure

§Integrations with other crates

§Non-fatal assertions with other test libraries

§Converting errors into test failures

§Features and dependencies

§Note on API stability

Re-exports§

Modules§

Macros§

Attribute Macros§

Crate test_that