mockforge-bench 0.3.3

Load and performance testing for MockForge
Documentation

MockForge Bench

Load and performance testing for MockForge that leverages OpenAPI specifications to generate realistic traffic patterns against real services.

Overview

MockForge Bench transforms MockForge from a mock server into a comprehensive load testing tool. By parsing OpenAPI specifications, it can automatically generate and execute load tests against real API endpoints using k6.

Features

  • Spec-driven testing: Automatically generate load tests from OpenAPI/Swagger specs
  • Multiple load scenarios: Support for constant, ramp-up, spike, stress, and soak tests
  • Operation filtering: Test specific endpoints or all operations
  • Realistic data: Generate request data from spec schemas or examples
  • Customizable thresholds: Set response time and error rate thresholds
  • Detailed reporting: Per-operation metrics and summary reports
  • Script generation: Generate k6 scripts for manual execution or CI/CD integration

Prerequisites

  • k6: The load testing tool must be installed 
    # macOS
brew install k6

# Linux
wget https://github.com/grafana/k6/releases/download/v0.48.0/k6-v0.48.0-linux-amd64.tar.gz
tar -xzf k6-v0.48.0-linux-amd64.tar.gz
sudo mv k6-v0.48.0-linux-amd64/k6 /usr/local/bin/

# Windows
choco install k6

Usage

Basic Load Test

mockforge bench --spec api.yaml --target https://api.example.com

Advanced Options

# Run a 5-minute load test with 100 virtual users
mockforge bench \
  --spec api.yaml \
  --target https://staging.api.com \
  --duration 5m \
  --vus 100 \
  --scenario ramp-up

# Test specific operations only
mockforge bench \
  --spec api.yaml \
  --target https://api.com \
  --operations "GET /users,POST /users" \
  --auth "Bearer token123"

# Generate k6 script without running
mockforge bench \
  --spec api.yaml \
  --target https://api.com \
  --generate-only \
  --script-output bench.js

# Custom thresholds and headers
mockforge bench \
  --spec api.yaml \
  --target https://api.com \
  --threshold-percentile p99 \
  --threshold-ms 1000 \
  --max-error-rate 0.01 \
  --headers "X-API-Key:abc123,X-Client-ID:client456"

Load Scenarios

Constant Load

Maintains a steady number of virtual users throughout the test.

mockforge bench --spec api.yaml --target https://api.com --scenario constant --vus 50

Ramp-up (Default)

Gradually increases load to the target number of virtual users.

mockforge bench --spec api.yaml --target https://api.com --scenario ramp-up --vus 100

Spike Test

Simulates a sudden spike in load to test system resilience.

mockforge bench --spec api.yaml --target https://api.com --scenario spike --vus 200

Stress Test

Continuously increases load to find the breaking point.

mockforge bench --spec api.yaml --target https://api.com --scenario stress --vus 500

Soak Test

Maintains sustained load over an extended period to detect memory leaks and degradation.

mockforge bench --spec api.yaml --target https://api.com --scenario soak --duration 1h --vus 50

Output

Results are saved to the output directory (default: bench-results/):

bench-results/
├── k6-script.js          # Generated k6 script
├── summary.json          # Test results in JSON format
└── detailed-results.json # Detailed per-operation metrics

Terminal Output

MockForge Bench - Load Testing Mode
────────────────────────────────────────────────────────────

Specification: api.yaml
Target: https://api.example.com
Operations: 5 endpoints
Scenario: ramp-up
Duration: 120s

→ Loading OpenAPI specification...
✓ Specification loaded
→ Extracting API operations...
✓ Found 5 operations
→ Generating request templates...
✓ Request templates generated
→ Generating k6 load test script...
✓ k6 script generated
✓ Script written to: bench-results/k6-script.js
→ Executing load test...

Load Test Complete! ✓

Summary:
  Total Requests:       15,680
  Successful:           15,612 (99.57%)
  Failed:               68 (0.43%)

Response Times:
  Avg:                  156ms
  p95:                  356ms
  p99:                  523ms

  Throughput:           130.7 req/s

Results saved to: bench-results/

CI/CD Integration

GitHub Actions Example

name: API Load Test

on:
  pull_request:
  workflow_dispatch:

jobs:
  load-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Install k6
        run: |
          wget https://github.com/grafana/k6/releases/download/v0.48.0/k6-v0.48.0-linux-amd64.tar.gz
          tar -xzf k6-v0.48.0-linux-amd64.tar.gz
          sudo mv k6-v0.48.0-linux-amd64/k6 /usr/local/bin/

      - name: Install MockForge
        run: cargo install --path .

      - name: Run load test
        run: |
          mockforge bench \
            --spec api/openapi.yaml \
            --target ${{ secrets.STAGING_API_URL }} \
            --duration 2m \
            --vus 50 \
            --output bench-results/ \
            --auth "Bearer ${{ secrets.API_TOKEN }}"

      - name: Upload results
        uses: actions/upload-artifact@v3
        with:
          name: load-test-results
          path: bench-results/

API

BenchCommand

The main command structure:

use mockforge_bench::BenchCommand;
use std::path::PathBuf;

let cmd = BenchCommand {
    spec: PathBuf::from("api.yaml"),
    target: "https://api.example.com".to_string(),
    duration: "5m".to_string(),
    vus: 100,
    scenario: "ramp-up".to_string(),
    operations: None,
    auth: Some("Bearer token123".to_string()),
    headers: None,
    output: PathBuf::from("bench-results"),
    generate_only: false,
    script_output: None,
    threshold_percentile: "p95".to_string(),
    threshold_ms: 500,
    max_error_rate: 0.05,
    verbose: false,
};

cmd.execute().await?;

Architecture

┌─────────────────────────────────────────┐
│  OpenAPI Spec                           │
└─────────────────┬───────────────────────┘
                  │
                  ▼
┌─────────────────────────────────────────┐
│  SpecParser                             │
│  - Parse OpenAPI spec                   │
│  - Extract operations                   │
└─────────────────┬───────────────────────┘
                  │
                  ▼
┌─────────────────────────────────────────┐
│  RequestGenerator                       │
│  - Generate request templates           │
│  - Extract parameters and bodies        │
└─────────────────┬───────────────────────┘
                  │
                  ▼
┌─────────────────────────────────────────┐
│  K6ScriptGenerator                      │
│  - Generate k6 JavaScript               │
│  - Configure scenarios and thresholds   │
└─────────────────┬───────────────────────┘
                  │
                  ▼
┌─────────────────────────────────────────┐
│  K6Executor                             │
│  - Execute k6 with generated script     │
│  - Stream output and progress           │
└─────────────────┬───────────────────────┘
                  │
                  ▼
┌─────────────────────────────────────────┐
│  Reporter                               │
│  - Parse results                        │
│  - Display summary                      │
└─────────────────────────────────────────┘

Development

Building

cargo build --release -p mockforge-bench

Testing

cargo test -p mockforge-bench

Adding New Load Scenarios

Extend the LoadScenario enum in scenarios.rs:

pub enum LoadScenario {
    Constant,
    RampUp,
    Spike,
    Stress,
    Soak,
    YourNewScenario,  // Add here
}

impl LoadScenario {
    pub fn generate_stages(&self, duration_secs: u64, max_vus: u32) -> Vec<Stage> {
        match self {
            // ... existing scenarios
            Self::YourNewScenario => {
                // Define your custom load pattern
                vec![/* your stages */]
            }
        }
    }
}

Contributing

Contributions are welcome! Please:

  1. Fork the repository
  2. Create a feature branch
  3. Add tests for new functionality
  4. Submit a pull request

License

MIT OR Apache-2.0