Please check the build logs for more information.
See Builds for ideas on how to fix a failed build, or Metadata for how to configure docs.rs builds.
If you believe this is docs.rs' fault, open an issue.
test-fuzz
At a high-level, test-fuzz
is a convenient front end for afl.rs
. In more concrete terms, test-fuzz
is a collection of Rust macros and a Cargo subcommand that automate certain fuzzing-related tasks, most notably:
- generating a fuzzing corpus
- implementing a fuzzing harness
test-fuzz
accomplishes these (in part) using Rust's testing facilities. For example, to generate a fuzzing corpus, test-fuzz
records a target's arguments each time it is called during an invocation of cargo test
. Similarly, test-fuzz
implements a fuzzing harness as an additional test in a cargo-test
-generated binary. This tight integration with Rust's testing facilities is what motivates the name test
-fuzz
.
Contents
- Installation
- Usage
- Components
test-fuzz
package features- Auto-generated corpus files
- Environment variables
- Limitations
- Tips and tricks
Installation
Usage
Fuzzing with test-fuzz
is essentially three steps:*
-
Identify a fuzz target:
- Add the following
dependencies
to the target crate'sCargo.toml
file:= "1.0" = "1.0.2"
- Precede the target function with the
test_fuzz
macro:
- Add the following
-
Generate a corpus by running
cargo test
:$ cargo test
-
Fuzz your target by running
cargo test-fuzz
:$ cargo test-fuzz --target foo
* Some additional steps may be necessary following a reboot. AFL requires the following commands to be run as root:
-
Linux
|
-
OSX
SL=/System/Library; PL=com.apple.ReportCrash
Components
test_fuzz
macro
Preceding a function with the test_fuzz
macro indicates that the function is a fuzz target.
The primary effects of the test_fuzz
macro are:
- Add instrumentation to the target to serialize its arguments and write them to a corpus file each time the target is called. The instrumentation is guarded by
#[cfg(test)]
so that corpus files are generated only when running tests (however, seeenable_in_production
below). - Add a test to read and deserialize arguments from standard input and apply the target to them. The test checks an environment variable, set by
cargo test-fuzz
, so that the test does not block trying to read from standard input during a normal invocation ofcargo test
. The test is enclosed in a module to reduce the likelihood of a name collision. Currently, the name of the module istarget_fuzz
, wheretarget
is the name of the target (however, seerename
below).
Arguments
-
bounds = "where_predicates"
- Imposewhere_predicates
(e.g., trait bounds) on the struct used to serialize/deserialize arguments. This may be necessary, e.g., if a target's argument type is an associated type. For an example, see associated_type.rs in this repository. -
concretize = "parameters"
- Useparameters
as the target's type parameters when fuzzing. Example:Note: The target's arguments must be serializable for every instantiation of its type parameters. But the target's arguments are required to be deserializable only when the target is instantiated with
parameters
. -
concretize_impl = "parameters"
- Useparameters
as the target'sSelf
type parameters when fuzzing. Example:Note: The target's arguments must be serializable for every instantiation of its
Self
type parameters. But the target's arguments are required to be deserializable only when the target'sSelf
is instantiated withparameters
. -
convert = "X, Y"
- When serializing the target's arguments, convert values of typeX
to typeY
usingY
's implementation ofFrom<X>
. When deserializing, convert those values back to typeX
usingY
's implementation of the non-standard traittest_fuzz::Into<X>
.That is, use of
convert = "X, Y"
should be accompanied by the following implementations:The definition of
test_fuzz::Into
is identical to that ofstd::conversion::Into
. The reason for using a non-standard trait is to avoid conflicts that could arise from blanket implementations of standard traits. -
enable_in_production
- Generate corpus files when not running tests, provided the environment variableTEST_FUZZ_WRITE
is set. The default is to generate corpus files only when running tests, regardless of whetherTEST_FUZZ_WRITE
is set. When running a target from outside its package directory, setTEST_FUZZ_MANIFEST_PATH
to the path of the package'sCargo.toml
file.WARNING: Setting
enable_in_production
could introduce a denial-of-service vector. For example, setting this option for a function that is called many times with different arguments could fill up the disk. The check ofTEST_FUZZ_WRITE
is meant to provide some defense against this possibility. Nonetheless, consider this option carefully before using it. -
execute_with = "function"
- Rather than call the target directly:- construct a closure of type
FnOnce() -> R
, whereR
is the target's return type, so that calling the closure calls the target; - call
function
with the closure.
Calling the target in this way allows
function
to set up the call's environment. This can be useful, e.g., for fuzzing Substrate externalities. - construct a closure of type
-
no_auto_generate
- Do not try to auto-generate corpus files for the target. -
only_concretizations
- Record the target's concretizations when running tests, but do not generate corpus files and do not implement a fuzzing harness. This can be useful when the target is a generic function, but it is unclear what type parameters should be used for fuzzing.The intended workflow is: enable
only_concretizations
, then runcargo test
followed bycargo test-fuzz --display-concretizations
. One of the resulting concretizations might be usable asconcretize
'sparameters
. Similarly, a concretization resulting fromcargo test-fuzz --display-imply-concretizations
might be usable asconcretize_impl
'sparameters
.Note, however, that just because a target was concretized with certain parameters during tests, it does not imply the target's arguments are serializable/deserializable when so concretized. The results of
--display-concretizations
/--display-impl-concretizations
are merely suggestive. -
rename = "name"
- Treat the target as though its name isname
when adding a module to the enclosing scope. Expansion of thetest_fuzz
macro adds a module definition to the enclosing scope. Currently, the module is namedtarget_fuzz
, wheretarget
is the name of the target. Use of this option causes the module to instead be be namedname_fuzz
. Example:// Without the use of `rename`, a name collision and compile error would result.
test_fuzz_impl
macro
Whenever the test_fuzz
macro is used in an impl
block,
the impl
must be preceded with the test_fuzz_impl
macro. Example:
The reason for this requirement is as follows. Expansion of the test_fuzz
macro adds a module definition to the enclosing scope. However, a module definition cannot appear inside an impl
block. Preceding the impl
with the test_fuzz_impl
macro causes the module to be added outside the impl
block.
If you see an error like the following, it likely means a use of the test_fuzz_impl
macro is missing:
error: module is not supported in `trait`s or `impl`s
test_fuzz_impl
currently has no options.
cargo test-fuzz
command
The cargo test-fuzz
command is used to interact with fuzz targets, and to manipulate their corpora, crashes, hangs, and work queues. Example invocations include:
-
List fuzz targets
cargo test-fuzz --list
-
Display target
foo
's corpuscargo test-fuzz --target foo --display-corpus
-
Fuzz target
foo
cargo test-fuzz --target foo
-
Replay crashes found for target
foo
cargo test-fuzz --target foo --replay-crashes
Usage
cargo test-fuzz [OPTIONS] [-- <args>...]
Args
<args>... Arguments for the fuzzer
Options
--backtrace
Display backtraces
--consolidate
Move one target's crashes, hangs, and work queue to its corpus; to consolidate all
targets, use --consolidate-all
--display-concretizations
Display concretizations
--display-corpus
Display corpus using uninstrumented fuzz target; to display with instrumentation, use
--display-corpus-instrumented
--display-crashes
Display crashes
--display-hangs
Display hangs
--display-impl-concretizations
Display `impl` concretizations
--display-queue
Display work queue
--exact
Target name is an exact name rather than a substring
--features <FEATURES>...
Space or comma separated list of features to activate
-h, --help
Print help information
--list
List fuzz targets
--manifest-path <PATH>
Path to Cargo.toml
--no-default-features
Do not activate the `default` feature
--no-instrumentation
Compile without instrumentation (for testing build process)
--no-run
Compile, but don't fuzz
--no-ui
Disable user interface
-p, --package <PACKAGE>
Package containing fuzz target
--persistent
Enable persistent mode fuzzing
--pretty-print
Pretty-print debug output when displaying/replaying
--replay-corpus
Replay corpus using uninstrumented fuzz target; to replay with instrumentation, use
--replay-corpus-instrumented
--replay-crashes
Replay crashes
--replay-hangs
Replay hangs
--replay-queue
Replay work queue
--reset
Clear fuzzing data for one target, but leave corpus intact; to reset all targets, use
--reset-all
--resume
Resume target's last fuzzing session
--run-until-crash
Stop fuzzing once a crash is found
--target <TARGET>
String that fuzz target's name must contain
--test <NAME>
Integration test containing fuzz target
--timeout <TIMEOUT>
Number of milliseconds to consider a hang when fuzzing or replaying (equivalent to `--
-t <timeout>` when fuzzing)
-V, --version
Print version information
dont_care!
macro
The dont_care!
macro can be used to implement serde::Serialize
/serde::Deserialize
for types that are easy to construct and whose values you do not care to record. Intuitively, dont_care!($ty, $expr)
says:
- Skip values of type
$ty
when serializing. - Initialize values of type
$ty
with$expr
when deserializing.
More specifically, dont_care!($ty, $expr)
expands to the following:
If $ty
is a unit struct, then $expr
can be be omitted. That is, dont_care!($ty)
is equivalent to dont_care!($ty, $ty)
.
Warning: dont_care!
is provided for convenience and may be removed in future versions of test-fuzz
.
test-fuzz
package features
The features in this section apply to the test-fuzz
package as a whole. Enable them in test-fuzz
's dependency specification as described in the The Cargo Book. For example, to enable the auto_concretize
feature, use:
= { = "1.0.2", = ["auto_concretize"] }
The test-fuzz
package currently supports the following features:
-
auto_concretize
- When this feature is enabled,test-fuzz
tries to inferimpl
and non-impl
concretizations. Success requires that a target be called with exactly oneimpl
concretization and exactly one non-impl
concretization during tests. Success is not guaranteed by these conditions, however.The implementation of
auto_concretize
uses the unstable language featureproc_macro_span
. So enablingauto_concretize
requires that targets be built with a nightly compiler. -
Serde formats -
test-fuzz
can serialize target arguments in multiple Serde formats. The following are the features used to select a format. Note that if a format other than the default is selected, thendefault-feature = false
must be specified.
Auto-generated corpus files
cargo-test-fuzz
can auto-generate values for types that implement certain traits. If all of a target's argument types implement such traits, cargo-test-fuzz
can auto-generate corpus files for the target.
The traits that cargo-test-fuzz
currently supports and the values generated for them are as follows:
Trait(s) | Value(s) |
---|---|
Bounded |
T::min_value() , T::max_value() |
Bounded + Add + One |
T::min_value() + T::one() |
Bounded + Add + Div + Two |
T::min_value() / T::two() + T::max_value() / T::two() |
Bounded + Add + Div + Two + One |
T::min_value() / T::two() + T::max_value() / T::two() + T::one() |
Bounded + Sub + One |
T::max_value() - T::one() |
Default |
T::default() |
Key
Add
-core::ops::Add
Bounded
-num_traits::bounds::Bounded
Default
-std::default::Default
Div
-core::ops::Div
One
-num_traits::One
Sub
-core::ops::Sub
Two
-test_fuzz::runtime::traits::Two
(essentiallyAdd + One
)
Environment variables
-
TEST_FUZZ_LOG
- During macro expansion:- If
TEST_FUZZ_LOG
is set to1
, write all instrumented fuzz targets and module definitions to standard output. - If
TEST_FUZZ_LOG
is set to a crate name, write that crate's instrumented fuzz targets and module definitions to standard output.
This can be useful for debugging.
- If
-
TEST_FUZZ_MANIFEST_PATH
- When running a target from outside its package directory, find the package'sCargo.toml
file at this location. One may need to set this environment variable whenenable_in_production
is used. -
TEST_FUZZ_WRITE
- Generate corpus files when not running tests for those targets for whichenable_in_production
is set.
Limitations
-
Clonable arguments - A target's arguments must implement the
Clone
trait. The reason for this requirement is that the arguments are needed in two places: in atest-fuzz
-internal function that writes corpus files, and in the body of the target function. To resolve this conflict, the arguments are cloned before being passed to the former. -
Serializable / deserializable arguments - In general, a target's arguments must implement the
serde::Serialize
andserde::Deserialize
traits, e.g., by deriving them. We say "in general" becausetest-fuzz
knows how to handle certain special cases that wouldn't normally be serializable/deserializable. For example, an argument of type&str
is converted toString
when serializing, and back to a&str
when deserializing. See alsoconcretize
andconcretize_impl
above. -
Global variables - The fuzzing harnesses that
test-fuzz
implements do not initialize global variables. Whileexecute_with
provides some remedy, it is not a complete solution. In general, fuzzing a function that relies on global variables requires ad-hoc methods. -
convert
andconcretize
/concretize_impl
- These options are incompatible in the following sense. If a fuzz target's argument type is a type parameter,convert
will try to match the type parameter, not the type to which it is concretized. Supporting the latter would seem to require simulating type substitution as the compiler would perform it. However, this is not currently implemented.
Tips and tricks
-
#[cfg(test)]
is not enabled for integration tests. If your target is tested only by integration tests, then consider usingenable_in_production
andTEST_FUZZ_WRITE
to generate a corpus. (Note the warning accompanyingenable_in_production
, however.) -
If you know the package in which your target resides, passing
-p <package>
tocargo test
/cargo test-fuzz
can significantly reduce build times. Similarly, if you know your target is called from only one integration test, passing--test <name>
can reduce build times. -
Rust won't allow you to implement
serde::Serialize
for other repositories' types. But you may be able to patch other repositories to make their types serializeble. Also,cargo-clone
can be useful for grabbing dependencies' repositories.