# Performance Runbook
This runbook documents how to reproduce performance evidence and apply rollout gates.
## Prerequisites
- Build binary:
- `cargo build --bin ir`
- Start from repo root.
## Reproducible Commands
1. Mixed-workload matrix:
- `bash scripts/mixed_workload_matrix.sh`
2. Soak + failure injection:
- `bash scripts/soak_failure_check.sh`
3. Concurrent ingest/query stress:
- `bash scripts/concurrent_stress.sh`
4. Runtime knob sweep:
- `bash scripts/tuning_sweep.sh`
5. Release gate:
- `bash scripts/release_gate.sh`
## Key Outputs
- `artifacts/mixed_workload_report.json`
- `artifacts/soak_failure_report.json`
- `artifacts/concurrent_stress_report.json`
- `artifacts/tuning_sweep_report.json`
- `artifacts/release_gate_report.json`
## Runtime Tuning Knobs
The `ir query` command supports the following environment knobs:
- `IR_QUERY_MORSEL_SIZE` (default `256`)
- `IR_QUERY_PARALLEL_WORKERS` (default `0` = adaptive)
- `IR_QUERY_SCAN_LIMIT_MULTIPLIER` (default `64`)
- `IR_QUERY_SCAN_MIN` (default `512`)
Recommended baseline from sweep:
- `IR_QUERY_MORSEL_SIZE=256`
- `IR_QUERY_PARALLEL_WORKERS=0`
- `IR_QUERY_SCAN_LIMIT_MULTIPLIER=64`
- `IR_QUERY_SCAN_MIN=512`
## Rollout Thresholds
Release gate (`release_gate.sh`) evaluates two readiness levels:
### Alpha readiness
- `error_rate <= 0.01`
- `p95 <= 10000` micros
- `p99 <= 15000` micros
- `concurrent ingest p50 >= 50000` events/sec
- `mixed burst ingest >= 200000` events/sec
- `soak recover_failures == 0`
### Beta readiness
- `error_rate <= 0.005`
- `p95 <= 5000` micros
- `p99 <= 8000` micros
- `concurrent ingest p50 >= 70000` events/sec
- `mixed burst ingest >= 250000` events/sec
- `soak recover_failures == 0`
## Troubleshooting
- Release gate fails due to missing artifacts:
- Re-run required scripts listed in "Reproducible Commands".
- Latency regression in sweep:
- Start from baseline knobs and re-run `tuning_sweep.sh`.
- Failure injection recovery failures:
- Re-run `soak_failure_check.sh` and inspect `artifacts/soak_failure_logs/`.
- Concurrent stress degradation:
- Inspect `artifacts/concurrent_stress_logs/` and compare against mixed-workload report.