Struct hyperpom::core::Worker

pub struct Worker<L: Loader, LD, GD> { /* private fields */ }

A fuzzing worker

Role of Fuzzing Workers in the Fuzzer

Workers are core components of the fuzzing process and each of them operates in its own dedicated thread. A worker primarily setup and manages fuzzing-related operations, but the actual execution is handled by an Executor instance.

                                    +--------------------+
                                    |                    |
                                    |      HYPERPOM      |
                                    |                    |
                                    +---------++---------+
                                              ||
                                              ||
          +-----------------------+-----------++-----------+-----------------------+
          |                       |                        |                       |
          |                       |                        |                       |
 +--------+--------+     +--------+--------+      +--------+--------+     +--------+--------+
 |     WORKER      |     |     WORKER      |      |     WORKER      |     |     WORKER      |
 |                 |     |                 |      |                 |     |                 |
 | +-------------+ |     | +-------------+ |      | +-------------+ |     | +-------------+ |
 | |             | |     | |             | |      | |             | |     | |             | |
 | |  EXECUTOR   | |     | |  EXECUTOR   | |      | |  EXECUTOR   | |     | |  EXECUTOR   | |
 | |             | |     | |             | |      | |             | |     | |             | |
 | +-------------+ |     | +-------------+ |      | +-------------+ |     | +-------------+ |
 +-----------------+     +-----------------+      +-----------------+     +-----------------+

Before fuzzing actually starts, workers go through an initialization phase that performs the following operations:

• creation of the worker’s working directory;
• calling the Executor::init function (sets up the initial address space, registers, etc.);
• loading the initial corpus.

Then, when Worker::run is called, the following operations are performed in a loop:

• restoring registers and the virtual address space using snapshots;
• resetting coverage and backtrace information;
• loading a testcase and mutates it;
• running the Loader::pre_exec hook;
• running the testcase using Executor::vcpu_run;
• running the Loader::post_exec hook;
• checking if a crash or a timeout occured;
  • if a crash occured, rerun the testcase with the backtrace hooks enabled and check if the crash is already known;
  • if it’s a new crash, store it;
• checking if new paths have been covered by the current testcase;
  • if new paths have been covered, update the GlobalCoverage object and add the testcase to the corpus.

Switching Between Coverage and Backtrace Hooks

Handling a hook is very expensive because of the context switch between the guest VM and the hypervisor. The fewer number of hooks we have to handle, the better the performances will be.

Hyperpom implements different hook types, with each their own drawbacks that may or may not be mitigable.

• Tracer hooks are not taken into account here, because we are not expecting tracing to be enabled while fuzzing.
• Exit hooks stop the execution altogether, so it’s essentially 0 cost.
• Custom hooks have to be executed everytime, there’s not much we can do here.
• Coverage hooks is one of the most expensive hook type, because it is applied to branch and comparison instructions on the whole binary. As explained in GlobalCoverage, to reduce the performance hit, these hooks are removed as soon as the path is covered, making them effectively one-shot hooks that won’t impact subsequent iterations.

Which leaves us with backtrace hooks. These hooks are also expensive since they are placed on function entries and exits. However, we don’t need them for every iteration, we’re only interested in getting the backtrace when a crash occurs.

The solution implemented in this fuzzer is to use two separate address spaces. Both have the same target binary loaded, the custom hooks applied, etc. But only one has the covrage hooks, while the other has the backtrace hooks. During normal execution, the address space with coverage hooks is used, but when a crash occurs, we switch to the one with the backtrace hooks. We compute the backtrace and see if the crash is stable, before switch back to coverage hooks.

The obvious downside is how much memory we’re using. Since we have two address spaces and their corresponding snapshots, we’re essentially mutliplying memory usage by four for a single worker. However, we don’t need to apply and remove hooks at every crash occurence, which can become costly, especially for large binaries. Hopefully, binaries that require huge memory allocations are uncommon enough that the chosen solution won’t be a limitation.