# Performance Benchmarks
This document describes how to run and interpret performance benchmarks for pg-api.
## Quick Start
### Running Benchmarks
```bash
# Run all benchmarks
cargo bench
# Run specific benchmark
cargo bench single_query
# Generate HTML reports (saved to target/criterion/)
cargo bench -- --verbose
```
### Running Load Tests
```bash
# Install dependencies
pip install locust requests
# Run standalone load test
python scripts/load_test.py --standalone
# Run with Locust UI
locust -f scripts/load_test.py --host=http://localhost:8580
# Open browser: http://localhost:8089
```
## Benchmark Categories
### 1. Single Query Performance
Tests the performance of individual query execution.
**Metrics:**
- Throughput (queries/second)
- Latency (p50, p95, p99)
- Query size impact
**Expected Results:**
- Simple SELECT: < 1ms
- Complex JOIN: < 10ms
- Large result set: < 100ms
### 2. Batch Query Performance
Tests batch processing capabilities.
**Metrics:**
- Batch size vs throughput
- Overhead per query in batch
- Maximum efficient batch size
**Expected Results:**
- 10 queries: < 5ms total
- 100 queries: < 50ms total
- 1000 queries: < 500ms total
### 3. JSON Serialization
Tests JSON encoding/decoding performance.
**Metrics:**
- Serialization speed (MB/s)
- Memory usage
- CPU utilization
**Expected Results:**
- 1KB payload: < 0.1ms
- 1MB payload: < 10ms
- 10MB payload: < 100ms
### 4. Connection Pool
Tests connection pool efficiency.
**Metrics:**
- Connection acquisition time
- Pool saturation behavior
- Connection reuse efficiency
**Expected Results:**
- Acquire connection: < 1ms
- Release connection: < 0.1ms
- Pool overhead: < 5%
### 5. Rate Limiting
Tests rate limiter performance impact.
**Metrics:**
- Check overhead per request
- Memory usage per account
- Accuracy of rate limiting
**Expected Results:**
- Rate check: < 0.01ms
- Memory per account: < 1KB
- Accuracy: 99.9%
## Load Testing Scenarios
### Scenario 1: Normal Load
- Users: 100 concurrent
- Request rate: 1000 req/s
- Duration: 5 minutes
```python
# Locust configuration
users = 100
spawn_rate = 10
run_time = "5m"
```
### Scenario 2: Peak Load
- Users: 1000 concurrent
- Request rate: 10,000 req/s
- Duration: 15 minutes
```python
# Locust configuration
users = 1000
spawn_rate = 50
run_time = "15m"
```
### Scenario 3: Sustained Load
- Users: 500 concurrent
- Request rate: 5000 req/s
- Duration: 1 hour
```python
# Locust configuration
users = 500
spawn_rate = 25
run_time = "1h"
```
## Performance Baseline
Based on testing with standard hardware (8 CPU cores, 16GB RAM):
### Query Performance
| Simple SELECT | 0.8ms | 2ms | 5ms | 10ms |
| Complex JOIN | 5ms | 15ms | 30ms | 50ms |
| Aggregation | 10ms | 30ms | 60ms | 100ms |
| Batch (10) | 5ms | 12ms | 25ms | 40ms |
| Batch (100) | 40ms | 80ms | 150ms | 200ms |
### Throughput
| Light | 1,000 | 10 | 10% | 200MB |
| Normal | 5,000 | 100 | 40% | 500MB |
| Heavy | 10,000 | 500 | 70% | 1GB |
| Peak | 15,000 | 1000 | 90% | 2GB |
### Connection Pool
| 10 | 500 | 2ms | 95% |
| 25 | 1,500 | 1.5ms | 97% |
| 50 | 3,000 | 1ms | 98% |
| 100 | 5,000 | 0.8ms | 99% |
## Optimization Tips
### Database Optimization
1. **Indexes**: Ensure proper indexes on frequently queried columns
2. **Connection Pooling**: Size pool based on `CPU cores * 2 + disk spindles`
3. **Query Planning**: Use `EXPLAIN ANALYZE` to optimize slow queries
4. **Partitioning**: Consider table partitioning for large datasets
### Application Optimization
1. **Caching**: Implement query result caching for read-heavy workloads
2. **Batch Processing**: Group related queries to reduce round trips
3. **Async Processing**: Use async/await effectively
4. **Resource Limits**: Set appropriate timeouts and limits
### Infrastructure Optimization
1. **CPU**: Scale cores for compute-intensive queries
2. **Memory**: Increase RAM for better caching
3. **Network**: Use low-latency connections
4. **Storage**: Use SSDs for database storage
## Monitoring Performance
### Key Metrics to Track
- **Response Time**: p50, p95, p99 latencies
- **Throughput**: Requests per second
- **Error Rate**: Failed requests percentage
- **Resource Usage**: CPU, memory, network I/O
- **Database Metrics**: Active connections, query time, lock waits
### Tools
```bash
# System monitoring
htop # CPU and memory
iotop # Disk I/O
iftop # Network I/O
# Database monitoring
pg_stat_statements # Query statistics
pg_stat_activity # Active connections
explain analyze # Query planning
# Application monitoring
cargo bench # Rust benchmarks
locust # Load testing
prometheus # Metrics collection
grafana # Visualization
```
## Benchmark Automation
### CI/CD Integration
Add to `.gitlab-ci.yml`:
```yaml
benchmark:
stage: test
script:
- cargo bench --no-fail-fast
- python scripts/load_test.py --standalone
artifacts:
paths:
- target/criterion/
expire_in: 30 days
only:
- main
- merge_requests
```
### Regression Detection
```bash
# Compare benchmark results
cargo bench -- --baseline main
# Set performance thresholds
cargo bench -- --save-baseline current
cargo bench -- --baseline current --strict
```
## Troubleshooting
### High Latency
1. Check database query performance
2. Verify connection pool settings
3. Look for lock contention
4. Review network latency
### Low Throughput
1. Increase connection pool size
2. Enable query caching
3. Optimize slow queries
4. Scale horizontally
### Memory Issues
1. Check for connection leaks
2. Review result set sizes
3. Implement pagination
4. Tune buffer sizes
## Example Benchmark Output
```
single_query/10 time: [784.21 ns 788.65 ns 793.54 ns]
thrpt: [12.60 Melem/s 12.68 Melem/s 12.75 Melem/s]
batch_queries/100 time: [45.234 ms 45.567 ms 45.912 ms]
thrpt: [2.178 Kelem/s 2.195 Kelem/s 2.211 Kelem/s]
json_serialization/1000 time: [1.2456 ms 1.2534 ms 1.2615 ms]
thrpt: [792.71 Kelem/s 797.82 Kelem/s 802.91 Kelem/s]
```
## Continuous Improvement
1. **Regular Benchmarking**: Run benchmarks weekly
2. **Track Trends**: Monitor performance over time
3. **Set Alerts**: Configure alerts for degradation
4. **Document Changes**: Note performance impact of changes
5. **Share Results**: Include in release notes