1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
# Benchmarking Guide
## Quick Start
```bash
cd bench/
# Fast smoke test (~30s)
cargo run -- --suite anti-detection
# Detection speed across all fixtures (~2 min)
cargo run -- --suite detection --instances 1 --max-concurrent 4
# Full benchmark suite (~20 min)
cargo run -- --suite all --instances 2 --max-concurrent 8
# Regression comparison
cargo run -- --suite regression --baseline bench-results-report.json --strict
```
## Output Files
Every run produces 6 files:
| File | Format | Use Case |
|------|--------|----------|
| `{prefix}-report.json` | JSON | Full raw data + aggregates |
| `{prefix}-aggregates.csv` | CSV | Summary statistics per suite×fixture |
| `{prefix}-observations.csv` | CSV | Per-iteration raw data |
| `{prefix}-report.md` | Markdown | GitHub PR comment / human review |
| `{prefix}-report.html` | HTML | Self-contained interactive report |
| `{prefix}-report.sarif` | SARIF | GitHub Security tab integration |
## Interpreting Results
### Summary Table Columns
| Column | Meaning |
|--------|---------|
| `ITERS` | Total iterations (success + failure) |
| `SUCC%` | Percentage of successful iterations |
| `MED(ms)` | Median elapsed time for successes |
| `STDEV` | Standard deviation of elapsed times |
| `CV` | Coefficient of variation (stddev / mean) |
### Reading Regression Analysis
```
--- Regression Analysis ---
turnstile/visible: +23.5% (45ms → 56ms) 🔴 REGRESSED
recaptcha/v3: -5.2% (120ms → 114ms) 🟢 OK
```
A regression is flagged when:
1. Delta > `--regression-threshold` (default: 20%)
2. AND statistical significance is detected (future: Mann-Whitney U test)
## Adding a New Fixture
1. Add `pub const FIXTURE_NAME: &str` to `bench/src/fixtures.rs`
2. Register in `bench/src/server.rs::fixtures()`
3. Rules:
- Fully self-contained (no external CDN)
- Include `<meta viewport>`
- Realistic DOM for `detect()` to identify
- Inline styles only
## Adding a New Suite
1. Create `bench/src/suites/my_suite.rs`
2. Export in `bench/src/suites/mod.rs`
3. Wire dispatch in `bench/src/main.rs`
4. Signature:
```rust
pub async fn run(
report: Arc<Mutex<Report>>,
cfg: &HarnessConfig,
) -> anyhow::Result<()>
```
## Fixture Catalog
See `bench/README.md` for the full catalog of 35+ fixtures organized by category.