1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128
//! Runt is a blazing fast, concurrent, and parallel snapshot testing framework.
//!
//! Snapshot testing involves running a command and testing its output against
//! an already known "golden" output file.
//! A runt test suite is defined using a `runt.toml` file.
//!
//! ## Installation
//!
//! To install the `runt` binary, simply run:
//! ```
//! cargo install runt
//! ```
//!
//! ## Testing Model
//! Runt's organizes tests using test suites.
//! At the minimum, a test suite needs to be specify the input file paths as
//! well as a command that is run for each input file.
//! For example, the following defines a test suite named "Cat tests" that
//! runs the command `cat {}` on every file `.txt` file in the directory
//! `cat-tests/`.
//! The `{}` is replaced by the path of the input file.
//! ```
//! [[tests]]
//! # Name for this test suite.
//! name = "Cat tests"
//! # Test paths can be globs or exact.
//! paths = [ "cat-test/*.txt" ]
//! # Command to run on each test file. {} is replaced with input name.
//! cmd = "cat {}"
//! # (Optional) Directory to store the generated .expect files.
//! expect_dir = "cat-out/"
//! # (Optional) Timeout for tests in seconds. Defaults to 1200 seconds.
//! timeout = 120
//! ```
//!
//! ## Running a Test Suite
//! Runt's command line interface is used to run and interact with a Runt
//! test suite.
//!
//! For example, we can run [Runt's own test suite][runt-suite].
//! From the directory containing the `runt.toml` file, run `runt`.
//! Runt will generate a summary by running all the test suites. It will also
//! print out the paths of the missing and the failing test suites:
//! ```text
//! ? Cat tests:cat-test/a.txt
//! ✗ Cat tests:cat-test/b.txt
//! ✗ Timeout test:timeout-test/input.txt (timeout)
//! ? Ls test:ls-test/input.txt
//! 1 passing / 2 failing / 2 missing
//! ```
//!
//! According to Runt, we have 2 failing and 2 missing tests.
//!
//! ## Filters
//!
//! A complete runt configuration might have hundreds of tests. Runt provides
//! two kinds of filter operations to select a subset of the test configuration.
//! - Pre-filters: The `--include` and `--exclude` flags can be used to select
//! tests names that match (or don't match) a particular regex. The regexes
//! are matched against the string `<suite-name>:<path>`.
//! - Post-filters: The `--only` flag can be used to print out the names of
//! tests with a specific exit condition (failing or missing). This is
//! particularly useful in conjuction with the `--diff` and `--save` flags.
//!
//!
//! For example, to view only the tests that failed, we can invoke:
//! ```bash
//! runt -o fail
//! ```
//! To which, `runt` will respond with:
//! ```text
//! ✗ Cat tests:cat-test/b.txt
//! ✗ Timeout test:timeout-test/input.txt (timeout)
//! 1 passing / 2 failing / 2 missing
//! ```
//! Note that Runt is still running all the tests. It simply suppressing the
//! output from the missing tests.
//!
//! If we only want to run the tests from the test suite `Cat tests`, we can
//! use the `-i` flag:
//! ```bash
//! runt -i 'Cat tests'
//! ```
//! To which, `runt` reports:
//! ```test
//! ✗ Cat tests:cat-test/b.txt
//! ? Cat tests:cat-test/a.txt
//! 0 passing / 1 failing / 1 missing
//! ```
//! Note that runt reports 0 passing tests because it is not running the test
//! suite with the previously passing tests.
//!
//! ## Viewing diffs and Saving .expect Files
//!
//! Under the hood, `runt` uses `.expect` files to test the outputs of running
//! a command.
//! The `expect_dir` option specifies the directory which contains the `.expect`
//! file (defaults to the directory containing the test path).
//! For example, we can view the `.expect` files for the "Cat tests" suite
//! under `cat-out`.
//!
//! Runt is also capable of showing diffs for failing and missing tests using
//! the `-d` flag.
//! For example, we can run:
//! ```bash
//! runt -i 'Cat tests' -d
//! ```
//! Runt generates diffs for both the missing test and the failing tests.
//!
//! In order to bless the diff as correct, we can use the `-s` flag to save
//! the output.
//! ```bash
//! runt -i 'Cat tests' -d -s
//! ```
//!
//! Both the `-d` and `-s` flags work with the filtering flags.
//!
//! ## Timeouts
//!
//! Test suites can require a default timeout for each individual test.
//! When left unspecified, Runt will use 20 minutes as the default.
//!
//! [runt-suite]: https://github.com/rachitnigam/runt/tree/master/cli-test
pub mod cli;
pub mod errors;
pub mod executor;
pub mod picker;
pub mod printer;