Crate trycmd[−][src]
Expand description
Snapshot testing for a herd of CLI tests
trycmd
is a test harness that will enumerate test case files and run them to verify the
results, taking inspiration from
trybuild and cram.
Which tool is right:
- Hand-written test cases: for peculiar circumstances
- assert_cmd: Test cases follow a certain pattern but special attention is needed in how to verify the results.
trycmd
: For running a lot of blunt tests (limited test predicates)- Particular attention is given to allow the test data to be pulled into documentation, like with mdbook
- cram: For cases agnostic of any programming language
Getting Started
To create a minimal setup, create a tests/cli_tests.rs
with
#[test]
fn cli_tests() {
trycmd::TestCases::new()
.case("tests/cmd/*.trycmd");
}
The test can be run with cargo test
. This will enumerate all .trycmd
files and run
them as test cases, failing if they do not pass.
To temporarily override the results, you can do:
#[test]
fn cli_tests() {
trycmd::TestCases::new()
.case("tests/cmd/*.trycmd")
// See Issue #314
.fail("tests/cmd/buggy-case.trycmd");
}
File Formats
Say you have tests/cmd/help.trycmd
(or help.toml
), trycmd
will look for:
tests/cmd/help.stdin
tests/cmd/help.stdout
tests/cmd/help.stderr
*.trycmd
A trycmd
file is just a command with arguments, with the arguments split with shlex.
The command is interpreted as bin.name
in a toml
file.
*.toml
As an alternative to .trycmd
, he toml
files give you a lot more control over how your command runs.
bin.name
: The name of the binary target fromCargo.toml
to be used to find the file path
*.stdin
Data to pass to stdin
.
- If not present, nothing will be written to
stdin
- If
binary = false
in*.toml
(the default), newlines will be normalized.
*.stdout
and *.stderr
Expected results for stdout
or stderr
.
- If not present, we’ll not verify the output
- If
binary = false
in*.toml
(the default), newlines will be normalized before comparing
Workflow
To generate snapshots, run
$ TRYCMD=dump cargo test --test cli_tests
This will write all of the .stdout
and .stderr
files in a dump/
directory.
You can then copy over to tests/cmd
the cases you want to test
To update snapshots, run
$ TRYCMD=overwrite cargo test --test cli_tests
This will overwrite any existing .stdout
and .stderr
file in tests/cmd
When iterating on a test, you can run:
cargo test --test cli_tests -- cli_tests trycmd=name1 trycmd=name2...
To filter the tests to those with name1
, name2
, etc in their file names.