Please check the build logs for more information.
See Builds for ideas on how to fix a failed build, or Metadata for how to configure docs.rs builds.
If you believe this is docs.rs' fault, open an issue.
test-fuzz
At a high-level, test-fuzz
is a convenient front end for afl.rs
. In more concrete terms, test-fuzz
is a collection of Rust macros and a Cargo subcommand that automate certain fuzzing-related tasks, most notably:
- generating a fuzzing corpus
- implementing a fuzzing harness
test-fuzz
accomplishes these (in part) using Rust's testing facilities. For example, to generate a fuzzing corpus, test-fuzz
records a target's arguments each time it is called during an invocation of cargo test
. Similarly, test-fuzz
implements a fuzzing harness as an additional test in a cargo-test
-generated binary. This tight integration with Rust's testing facilities is what motivates the name test
-fuzz
.
Contents
- Installation
- Usage
- Components
test-fuzz
Package Features- Auto-generated Corpus Files
- Environment Variables
- Limitations
- Tips and Tricks
Installation
Usage
Fuzzing with test-fuzz
is essentially three steps:*
-
Identify a fuzz target:
- Add the following
dependencies
to the target crate'sCargo.toml
file:= "1.0" = "1.0.0"
- Precede the target function with the
test_fuzz
macro:
- Add the following
-
Generate a corpus by running
cargo test
:$ cargo test
-
Fuzz your target by running
cargo test-fuzz
:$ cargo test-fuzz --target foo
* Some additional steps may be necessary following a reboot. AFL requires the following commands to be run as root:
-
Linux
|
-
OSX
SL=/System/Library; PL=com.apple.ReportCrash
Components
test_fuzz
macro
Preceding a function with the test_fuzz
macro indicates that the function is a fuzz target.
The primary effects of the test_fuzz
macro are:
- Add instrumentation to the target to serialize its arguments and write them to a corpus file each time the target is called. The instrumentation is guarded by
#[cfg(test)]
so that corpus files are generated only when running tests (however, seeenable_in_production
below). - Add a test to read and deserialize arguments from standard input and apply the target to them. The test checks an environment variable, set by
cargo test-fuzz
, so that the test does not block trying to read from standard input during a normal invocation ofcargo test
. The test is enclosed in a module to reduce the likelihood of a name collision. Currently, the name of the module istarget_fuzz
, wheretarget
is the name of the target (however, seerename
below).
Arguments
-
bounds = "where_predicates"
- Imposewhere_predicates
(e.g., trait bounds) on the struct used to serialize/deserialize arguments. This may be necessary, e.g., if a target's argument type is an associated type. For an example, see associated_type.rs in this repository. -
concretize = "parameters"
- Useparameters
as the target's type parameters when fuzzing. Example:Note: The target's arguments must be serializable for every instantiation of its type parameters. But the target's arguments are required to be deserializable only when the target is instantiated with
parameters
. -
concretize_impl = "parameters"
- Useparameters
as the target'sSelf
type parameters when fuzzing. Example:Note: The target's arguments must be serializable for every instantiation of its
Self
type parameters. But the target's arguments are required to be deserializable only when the target'sSelf
is instantiated withparameters
. -
convert = "X, Y"
- When serializing the target's arguments, convert values of typeX
to typeY
usingY
's implementation ofFrom<X>
. When deserializing, convert those values back to typeX
usingY
's implementation of the non-standard traittest_fuzz::Into<X>
.That is, use of
convert = "X, Y"
should be accompanied by the following implementations:The definition of
test_fuzz::Into
is identical to that ofstd::conversion::Into
. The reason for using a non-standard trait is to avoid conflicts that could arise from blanket implementations of standard traits. -
enable_in_production
- Generate corpus files when not running tests, provided the environment variableTEST_FUZZ_WRITE
is set. The default is to generate corpus files only when running tests, regardless of whetherTEST_FUZZ_WRITE
is set. When running a target from outside its package directory, setTEST_FUZZ_MANIFEST_PATH
to the path of the package'sCargo.toml
file.WARNING: Setting
enable_in_production
could introduce a denial-of-service vector. For example, setting this option for a function that is called many times with different arguments could fill up the disk. The check ofTEST_FUZZ_WRITE
is meant to provide some defense against this possibility. Nonetheless, consider this option carefully before using it. -
execute_with = "function"
- Rather than call the target directly:- construct a closure of type
FnOnce() -> R
, whereR
is the target's return type, so that calling the closure calls the target; - call
function
with the closure.
Calling the target in this way allows
function
to set up the call's environment. This can be useful, e.g., for fuzzing Substrate externalities. - construct a closure of type
-
no_auto_generate
- Do not try to auto-generate corpus files for the target. -
only_concretizations
- Record the target's concretizations when running tests, but do not generate corpus files and do not implement a fuzzing harness. This can be useful when the target is a generic function, but it is unclear what type parameters should be used for fuzzing.The intended workflow is: enable
only_concretizations
, then runcargo test
followed bycargo test-fuzz --display-concretizations
. One of the resulting concretizations might be usable asconcretize
'sparameters
. Similarly, a concretization resulting fromcargo test-fuzz --display-imply-concretizations
might be usable asconcretize_impl
'sparameters
.Note, however, that just because a target was concretized with certain parameters during tests, it does not imply the target's arguments are serializable/deserializable when so concretized. The results of
--display-concretizations
/--display-impl-concretizations
are merely suggestive. -
rename = "name"
- Treat the target as though its name isname
when adding a module to the enclosing scope. Expansion of thetest_fuzz
macro adds a module definition to the enclosing scope. Currently, the module is namedtarget_fuzz
, wheretarget
is the name of the target. Use of this option causes the module to instead be be namedname_fuzz
. Example:// Without the use of `rename`, a name collision and compile error would result.
test_fuzz_impl
macro
Whenever the test_fuzz
macro is used in an impl
block,
the impl
must be preceded with the test_fuzz_impl
macro. Example:
The reason for this requirement is as follows. Expansion of the test_fuzz
macro adds a module definition to the enclosing scope. However, a module definition cannot appear inside an impl
block. Preceding the impl
with the test_fuzz_impl
macro causes the module to be added outside the impl
block.
If you see an error like the following, it likely means a use of the test_fuzz_impl
macro is missing:
error: module is not supported in `trait`s or `impl`s
test_fuzz_impl
currently has no options.
cargo test-fuzz
command
The cargo test-fuzz
command is used to interact with fuzz targets, and to manipulate their corpora, crashes, hangs, and work queues. Example invocations include:
-
List fuzz targets
cargo test-fuzz --list
-
Display target
foo
's corpuscargo test-fuzz --target foo --display-corpus
-
Fuzz target
foo
cargo test-fuzz --target foo
-
Replay crashes found for target
foo
cargo test-fuzz --target foo --replay-crashes
Usage
cargo test-fuzz [FLAGS] [OPTIONS] [-- <args>...]
Args
<args>... Arguments for the fuzzer
Flags
--backtrace
Display backtraces
--consolidate
Move one target's crashes, hangs, and work queue to its corpus; to consolidate all
targets, use --consolidate-all
--display-concretizations
Display concretizations
--display-corpus
Display corpus using uninstrumented fuzz target; to display with instrumentation, use
--display-corpus-instrumented
--display-crashes
Display crashes
--display-hangs
Display hangs
--display-impl-concretizations
Display `impl` concretizations
--display-queue
Display work queue
--exact
Target name is an exact name rather than a substring
-h, --help
Print help information
--list
List fuzz targets
--no-default-features
Do not activate the `default` feature
--no-instrumentation
Compile without instrumentation (for testing build process)
--no-run
Compile, but don't fuzz
--no-ui
Disable user interface
--persistent
Enable persistent mode fuzzing
--pretty-print
Pretty-print debug output when displaying/replaying
--replay-corpus
Replay corpus using uninstrumented fuzz target; to replay with instrumentation, use
--replay-corpus-instrumented
--replay-crashes
Replay crashes
--replay-hangs
Replay hangs
--replay-queue
Replay work queue
--reset
Clear fuzzing data for one target, but leave corpus intact; to reset all targets, use
--reset-all
--resume
Resume target's last fuzzing session
--run-until-crash
Stop fuzzing once a crash is found
-V, --version
Print version information
Options
--features <FEATURES>... Space or comma separated list of features to activate
--manifest-path <PATH> Path to Cargo.toml
-p, --package <PACKAGE> Package containing fuzz target
--target <TARGET> String that fuzz target's name must contain
--test <NAME> Integration test containing fuzz target
--timeout <TIMEOUT> Number of milliseconds to consider a hang when fuzzing or
replaying (equivalent to `-- -t <timeout>` when fuzzing)
dont_care!
macro
The dont_care!
macro can be used to implement serde::Serialize
/serde::Deserialize
for types that are easy to construct and whose values you do not care to record. Intuitively, dont_care!($ty, $expr)
says:
- Skip values of type
$ty
when serializing. - Initialize values of type
$ty
with$expr
when deserializing.
More specifically, dont_care!($ty, $expr)
expands to the following:
If $ty
is a unit struct, then $expr
can be be omitted. That is, dont_care!($ty)
is equivalent to dont_care!($ty, $ty)
.
Warning: dont_care!
is provided for convenience and may be removed in future versions of test-fuzz
.
test-fuzz
Package Features
The features in this section apply to the test-fuzz
package as a whole. Enable them in test-fuzz
's dependency specification as described in the The Cargo Book. For example, to enable the auto_concretize
feature, use:
= { = "1.0.0", = ["auto_concretize"] }
The test-fuzz
package currently supports the following features:
-
auto_concretize
- When this feature is enabled,test-fuzz
tries to inferimpl
and non-impl
concretizations. Success requires that a target be called with exactly oneimpl
concretization and exactly one non-impl
concretization during tests. Success is not guaranteed by these conditions, however.The implementation of
auto_concretize
uses the unstable language featureproc_macro_span
. So enablingauto_concretize
requires that targets be built with a nightly compiler. -
Serde formats -
test-fuzz
can serialize target arguments in multiple Serde formats. The following are the features used to select a format. Note that if a format other than the default is selected, thendefault-feature = false
must be specified.
Auto-generated Corpus Files
cargo-test-fuzz
can auto-generate values for types that implement certain traits. If all of a target's argument types implement such traits, cargo-test-fuzz
can auto-generate corpus files for the target.
The traits that cargo-test-fuzz
currently supports and the values generated for them are as follows:
Trait(s) | Value(s) |
---|---|
Bounded |
T::min_value() , T::max_value() |
Bounded + Add + One |
T::min_value() + T::one() |
Bounded + Add + Div + Two |
T::min_value() / T::two() + T::max_value() / T::two() |
Bounded + Add + Div + Two + One |
T::min_value() / T::two() + T::max_value() / T::two() + T::one() |
Bounded + Sub + One |
T::max_value() - T::one() |
Default |
T::default() |
Key
Add
-core::ops::Add
Bounded
-num_traits::bounds::Bounded
Default
-std::default::Default
Div
-core::ops::Div
One
-num_traits::One
Sub
-core::ops::Sub
Two
-test_fuzz::runtime::traits::Two
(essentiallyAdd + One
)
Environment Variables
-
TEST_FUZZ_LOG
- During macro expansion:- If
TEST_FUZZ_LOG
is set to1
, write all instrumented fuzz targets and module definitions to standard output. - If
TEST_FUZZ_LOG
is set to a crate name, write that crate's instrumented fuzz targets and module definitions to standard output.
This can be useful for debugging.
- If
-
TEST_FUZZ_MANIFEST_PATH
- When running a target from outside its package directory, find the package'sCargo.toml
file at this location. One may need to set this environment variable whenenable_in_production
is used. -
TEST_FUZZ_WRITE
- Generate corpus files when not running tests for those targets for whichenable_in_production
is set.
Limitations
-
Clonable arguments - A target's arguments must implement the
Clone
trait. The reason for this requirement is that the arguments are needed in two places: in atest-fuzz
-internal function that writes corpus files, and in the body of the target function. To resolve this conflict, the arguments are cloned before being passed to the former. -
Serializable / deserializable arguments - In general, a target's arguments must implement the
serde::Serialize
andserde::Deserialize
traits, e.g., by deriving them. We say "in general" becausetest-fuzz
knows how to handle certain special cases that wouldn't normally be serializable/deserializable. For example, an argument of type&str
is converted toString
when serializing, and back to a&str
when deserializing. See alsoconcretize
andconcretize_impl
above. -
Global variables - The fuzzing harnesses that
test-fuzz
implements do not initialize global variables. No general purpose solution for this problem currently exists. So, to fuzz a function that relies on global variables usingtest-fuzz
, ad-hoc methods must be used.
Tips and tricks
-
#[cfg(test)]
is not enabled for integration tests. If your target is tested only by integration tests, then consider usingenable_in_production
andTEST_FUZZ_WRITE
to generate a corpus. (Note the warning accompanyingenable_in_production
, however.) -
If you know the package in which your target resides, passing
-p <package>
tocargo test
/cargo test-fuzz
can significantly reduce build times. -
Rust won't allow you to implement
serde::Serialize
for other repositories' types. But you may be able to patch other repositories to make their types serializeble. Also,cargo-clone
can be useful for grabbing dependencies' repositories.