## Criterion benchmarks for Tonic
### Running the benchmarks
From the root Tonic directory, `cargo bench`
After running, the reports can be found in `tonic/target/criterion/report/index.html`
[Gnuplot](http://www.gnuplot.info/) is required for graph generation. If gnuplot is not installed, Criterion will display: `Gnuplot not found, disabling plotting` at the console.
### Notes
1) Currently, these benchmarks only test the performance of constructing Tonic Requests and Responses, not over-the-wire throughput.
2) The `thrpt` value generated by Criterion is simply a measure of bytes consumed by the target function.
3) As we are not testing tonic-build compile time, the tests reference pre-compiled .rs files in 'benchmarks/compiled_protos'.
4) The original proto files are in the `proto` directory for reference.
5) This used the Criterion 3.0 `Criterion Group` functionality. Details here: https://docs.rs/criterion/0.3.0/criterion/
### Interpreting Results
Criterion is particularly useful for establishing a first-run baseline and then comparing after code-changes - e.g. `Performance has regressed` below.
```bash
Request_Response/request/100000
time: [2.7231 us 2.7588 us 2.7969 us]
thrpt: [33.298 GiB/s 33.758 GiB/s 34.200 GiB/s]
change:
time: [+16.073% +17.871% +19.980%] (p = 0.00 < 0.05)
thrpt: [-16.653% -15.162% -13.847%]
Performance has regressed.
Found 3 outliers among 100 measurements (3.00%)
1 (1.00%) high mild
2 (2.00%) high severe
```