API testing CLIs overview

Overview

This workspace ships three Rust API testing CLIs (api-rest, api-gql, api-test) plus the shared library crate api-testing-core. Behavioral parity with the legacy scripts remains the top priority: flags, defaults, exit codes, and on-disk artifacts (history files, reports, results).

Detailed parity specs live with each binary:

• crates/api-rest/README.md
• crates/api-gql/README.md
• crates/api-test/README.md

This README focuses on the shared repository layout, cross-CLI concepts, and the api-testing-core surface area.

Binaries and legacy script mapping

• api-rest (REST)
  • api-rest call (default) : parity with rest.sh
  • api-rest history : parity with rest-history.sh
  • api-rest report : parity with rest-report.sh
• api-gql (GraphQL)
  • api-gql call (default) : parity with gql.sh
  • api-gql history : parity with gql-history.sh
  • api-gql report : parity with gql-report.sh
  • api-gql schema : parity with gql-schema.sh
• api-test (suite runner)
  • api-test run : parity with api-test.sh
  • api-test summary : parity with api-test-summary.sh

Notes:

• api-test executes REST/GraphQL cases through shared core runners (no shelling out to scripts or the binaries).
• api-rest/api-gql also provide report-from-cmd as a Rust-only convenience for replaying saved call snippets.

api-testing-core scope

api-testing-core is a library crate used by all three CLIs. Key modules:

• config: setup dir discovery for REST/GraphQL configs.
• env_file: .env parsing + key normalization helpers.
• cli_*: shared CLI helpers (endpoint resolution, history I/O, report args, CLI utilities).
• history, report, markdown, redact, cmd_snippet: shared report/history rendering and snippet parsing.
• rest: request schema, runner, expect/cleanup logic, report rendering.
• graphql: schema/vars loading, auth/JWT resolution, runner, expect/allow-errors, report rendering, mutation detection.
• suite: suite schema v1, path resolution, filters, safety gates, auth integration, runner, cleanup, results, summary, JUnit.

Shared terminology

• Setup dir
  • Protocol-specific config directory.
  • Canonical locations: REST setup/rest, GraphQL setup/graphql.
• Config dir
  • CLI arg --config-dir <dir> that seeds setup-dir discovery.
  • Discovery searches upward for known config files; fallback uses the canonical setup/<tool> when applicable.
• Env preset
  • --env <name> selects a base URL from endpoints.env (+ optional .local overrides).
  • REST: REST_URL_<ENV_KEY>; GraphQL: GQL_URL_<ENV_KEY>.
  • If the value looks like http(s)://..., it is treated as a direct URL (like --url).
• Token/JWT profile
  • REST: --token <name> or REST_TOKEN_NAME selects REST_TOKEN_<NAME>.
  • GraphQL: --jwt <name> or GQL_JWT_NAME selects GQL_JWT_<NAME>.
  • REST fallback: ACCESS_TOKEN, then SERVICE_TOKEN if no profile is selected.
  • GraphQL fallback: ACCESS_TOKEN, then SERVICE_TOKEN if no profile is selected.
• History
  • REST: <setup_dir>/.rest_history, GraphQL: <setup_dir>/.gql_history.
  • Enabled by default, can be disabled, and supports rotation/size limits.
• Report
  • Markdown artifact (usually under <repo>/docs/) capturing request/operation, response, and optional assertions.
  • Redacts common secret fields by default, with opt-out flags for debugging.
• Suite
  • JSON manifest (version: 1) that drives api-test run and defines cases, defaults, auth, and cleanup.

Canonical repo layouts

The CLIs support the same layouts the legacy scripts assume.

Layout A: App repo with setup/ + tests/ (recommended)

<repo>/
  setup/
    rest/
      endpoints.env
      endpoints.local.env        # optional (local override)
      tokens.env
      tokens.local.env           # optional (local tokens; do not commit)
    graphql/
      endpoints.env
      endpoints.local.env        # optional (local override)
      jwts.env
      jwts.local.env             # optional (local tokens; do not commit)
      schema.env                 # optional (sets GQL_SCHEMA_FILE)
      schema.local.env           # optional (local override)
      schema.graphql             # or: schema.gql / schema.graphqls / api.graphql / api.gql
      operations/                # optional (login.graphql, shared ops, etc.)
  tests/
    api/
      suites/
        <name>.suite.json
  out/
    api-test-runner/             # suite runner output base dir

Layout B: Suites under setup/ (fallback)

api-test resolves --suite <name> to:

• <repo>/tests/api/suites/<name>.suite.json (preferred)
• <repo>/setup/api/suites/<name>.suite.json (fallback)

Layout C: Custom suites directory

• API_TEST_SUITES_DIR=<path> overrides the suites directory for --suite <name>.

Quickstart examples

api-rest

Run a request:

api-rest call --env staging setup/rest/requests/health.request.json

Write a report:

api-rest report --case health --request setup/rest/requests/health.request.json --run

Generate a report from a saved snippet:

api-rest history --command-only | api-rest report-from-cmd --stdin

api-gql

Run an operation:

api-gql call --env staging setup/graphql/operations/health.graphql

Write a report:

api-gql report --case health --op setup/graphql/operations/health.graphql --run

Resolve and print schema:

api-gql schema --cat

api-test

Run a suite (always emits results JSON to stdout):

api-test run --suite smoke

Use an explicit suite file:

api-test run --suite-file tests/api/suites/smoke.suite.json

Write results JSON + JUnit:

api-test run --suite smoke --out out/api-test-runner/results.json --junit out/api-test-runner/junit.xml

Render a Markdown summary:

api-test summary --in out/api-test-runner/results.json --out out/api-test-runner/summary.md

Suite runner behavior (high-level)

• Results JSON is always emitted to stdout. --out writes an additional copy.
• Output directory base defaults to <repo>/out/api-test-runner (override via API_TEST_OUTPUT_DIR).
• Each run creates <output_dir>/<run_id>/ where run_id is YYYYMMDD-HHMMSSZ.
• Per-case artifacts include <case>.response.json and <case>.stderr.log, referenced in results JSON.
• Exit code is 2 when any case fails; otherwise 0.
• Write safety is two-step:
  • Case must set allowWrite: true.
  • Writes must be enabled via --allow-writes, API_TEST_ALLOW_WRITES_ENABLED=true, or env: local.
• Optional suite auth can derive tokens from secret JSON in API_TEST_AUTH_JSON.
• Optional cleanup steps run after the main case; cleanup failures mark the case failed.

Suite runner environment variables

• API_TEST_OUTPUT_DIR: override the base output directory.
• API_TEST_SUITES_DIR: override the suites directory used by --suite.
• API_TEST_ALLOW_WRITES_ENABLED: enable write-capable cases.
• API_TEST_REST_URL: override REST base URL for all REST/rest-flow cases.
• API_TEST_GQL_URL: override GraphQL endpoint URL for all GraphQL cases.
• API_TEST_AUTH_JSON: credentials JSON used by suite auth (default key name).
• GITHUB_STEP_SUMMARY: when set, api-test summary appends Markdown output (disable via --no-github-summary).

Docs

• Docs index