phink 0.1.5

🐙 Phink, a ink! smart-contract property-based and coverage-guided fuzzer
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107

# Benchmarking

Benchmarking provides insights into Phink's performance in real-world scenarios, in order to vizualise its efficiency
and
fuzzing ink! smart contracts. Below are the benchmark results for various smart contracts, detailing
coverage, speed, corpus size, and the potential usage of `generate-seed`.
Each contract were fuzzed for maximum a day.
Statistics (especially *average speed*) are given for **one** core only. The coverage percent is calculated using the
number of
lines covered divided the number of reachable lines, as a percentage.

> ⚠️ The point of the benchmark is to demonstrate how much coverage is reachable within a day of fuzzing without doing
> proper seed creation. In a real fuzzing campaign, the developers would aim for 100% coverage, by creating seeds,
> adding `GenesisConfig` values, more (E2E) tests extracted with `seed-generator`, etc.

### Benchmarks

| Contract name | Coverage percent | Average speed (_execs/sec_)                       | AFL++ corpus size | Using Phink seed generation |
|---------------|------------------|---------------------------------------------------|-------------------|-----------------------------|
| abax_governor | **48%**          | 1500 (early phase) **and** 100 (late phase)       | 1639              | **NO** (no tests available) |
| erc1155       | **89%**          | 1300 (early phase phase) **and** 140 (late phase) | 949               | **YES** (without E2E)       |
| multisig      | **91%**          | 1400 (early phase phase) **and** 113 (late phase) | 1524              | **YES** (without E2E)       |

- Github for
  `abax_governor` : [AbaxFinance/dao-contracts/tree/main/src/contracts/abax_governor](https://github.com/AbaxFinance/dao-contracts/tree/main/src/contracts/abax_governor/)
- Github for
  `multisig` : [use-ink/ink-examples/blob/main/multisig/lib.rs](https://github.com/use-ink/ink-examples/blob/main/multisig/lib.rs)
- Github for
  `erc1155` : [use-ink/ink-examples/blob/main/erc1155/lib.rs](https://github.com/use-ink/ink-examples/blob/main/erc1155/lib.rs)

##### Dummy benchmark

The [dummy](https://github.com/srlabs/phink/blob/main/sample/dummy/lib.rs) benchmark involves a simple nested
if-condition. It acts as a reference to ensure that the fuzzer is
effectively coverage guided. The results for this benchmark are as follows:

* **Average speed**: 7,500 executions per second in average
* **Number of cores used**: 10
* **Time until invariant triggered**: 48 seconds
* **Stability**: 99.43%
* **Fuzzing origin**: false
* **Final corpus size**: 12 seeds

###### Dummy logic

The logic tested in the dummy benchmark can simply be represented that way:

```rust, ignore
if data.len() > 3 && data.len() < 7 {
    if data.chars().nth(0).unwrap() == 'f' {
        if data.chars().nth(1).unwrap() == 'u' {
            if data.chars().nth(2).unwrap() == 'z' {
                if data.chars().nth(3).unwrap() == 'z' {
                    self.forbidden_number = 42;
                }
            }
        }
    }
}
``` 

#### Contracts

###### ERC-1155

> The ERC-1155 contract is a standard for creating multiple token types within a single contract. It allows for the
> creation of both fungible and non-fungible tokens and enables batch transfers, making it easy to transfer multiple
> tokens at once.

###### Multisig Wallet

> The Multisig Wallet contract is a multi-owner wallet that requires a certain number of owners to agree on a
> transaction before it can be executed. Each owner can submit a transaction, and when enough owners confirm, it can be
> executed.

###### AbaxGovernor

> The Abax Governor contract is a governance contract that allows for staking of PSP22 tokens in exchange for
> non-transferrable PSP22Vault shares (votes). It enables users to propose and vote on proposals, with the number of
> shares held by a user determining their voting power.

### Explanation of terms

- **Coverage**: Represents the percentage of the code that have been executed during the fuzzing campaign. Higher
  coverage
  indicates more thorough testing (_the higher the better_).

- **Average speed (for 1 core)**: The number of executions per second that the fuzzer can handle on a single CPU core.
  As a reminder, one execution contains multiple calls up to
  `max_messages_per_exec`.

- **AFL++ corpus size**: The size of the corpus generated by AFL++ during fuzzing. A larger
  corpus implies a diverse set of inputs to test the contract.

- **generate-seed usage**: Indicates whether `generate-seed` was used to seed the initial tests. This depends if the
  contract include tests or not.

### Environment details

- **CPU**: AMD EPYC 7282 16-Cores
- **Operating System**: Linux 5.4.0-189-generic #209-Ubuntu x86_64
- **Phink Version**: 0.1.4

### Contributing to the benchmarks

We encourage contributions to our benchmarks! If you have a contract you would like to see benchmarked, please submit a
pull request to our repository.