tarn 0.7.0

CLI-first API testing tool
Documentation

Tests are .tarn.yaml files — YAML in, structured JSON out. Single binary, zero dependencies. Designed for the AI coding workflow: an LLM writes tests, runs tarn run --format json, parses results, iterates.

name: Health check
steps:
  - name: GET /health
    request:
      method: GET
      url: "{{ env.base_url }}/health"
    assert:
      status: 200

$ tarn run
 TARN  Running tests/health.tarn.yaml

 ● Health check

   ✓ GET /health (4ms)

 Results: 1 passed (15ms)

Why Tarn?

  • 50% fewer tokens than equivalent TypeScript/Python tests — faster LLM generation, lower cost
  • Structured JSON output with request/response on failures — machines parse it, not regex
  • Single binary — curl | sh install, runs in any CI, no runtime needed
  • MCP server — direct integration with Claude Code, opencode, Cursor, Windsurf
  • Everything you need — REST, GraphQL, captures, cookies, multipart, includes, polling, Lua scripting, parallel execution, 7 output formats

Install

# One-liner (macOS / Linux)
curl -fsSL https://raw.githubusercontent.com/NazarKalytiuk/hive/main/install.sh | sh

# Install to a custom directory
TARN_INSTALL_DIR="$HOME/.local/bin" curl -fsSL https://raw.githubusercontent.com/NazarKalytiuk/hive/main/install.sh | sh

# Or build/install from source
cargo install --git https://github.com/NazarKalytiuk/hive.git --bin tarn

Binaries for macOS (Intel & Apple Silicon), Linux (amd64 & arm64), and Windows (amd64 zip) are published on the releases page. Each release also includes hive-checksums.txt for SHA256 verification and a generated tarn.rb Homebrew formula artifact.

Container path:

  • ghcr.io/<owner>/tarn:<tag> from the release workflow

Installer notes:

  • install.sh verifies the downloaded archive against hive-checksums.txt
  • TARN_INSTALL_DIR controls the install destination
  • HIVE_INSTALL_DIR is still accepted as a backward-compatible alias during the rename transition
  • Manual verification also works with shasum -a 256 -c hive-checksums.txt

Quick Start

tarn init                              # scaffold tests/health + advanced examples/ templates
# Edit tarn.env.yaml to point at your API, or start one on http://localhost:3000
tarn run                               # run all tests
tarn run tests/health.tarn.yaml        # run the scaffolded test directly
tarn fmt --check                       # verify canonical YAML formatting
tarn run --env staging                 # use staging environment
tarn run --format json                 # structured output for LLM/CI
tarn run --only-failed                 # show only failing tests in the output
tarn run --no-progress                 # disable streaming progress (batch dump at end)
tarn run --watch                       # re-run on file changes
tarn run --parallel                    # run files in parallel
tarn list --tag smoke                  # inspect matching tests without running them

Hello World

Want a fully local demo path from this repo?

PORT=3000 cargo run -p demo-server &
cargo run -p tarn -- run examples/demo-server/hello-world.tarn.yaml

This exercises a local API with no external network dependency. There are more local scenarios in examples/demo-server/ for redirects, cookies, forms, error responses, and authenticated CRUD flows.

Table of Contents

Docs Index

Canonical project docs live in docs/INDEX.md.

If you are looking for product direction or comparisons, start with:

For AI-assisted workflows, see also:

For editor integrations:

  • docs/TARN_LSP.md — tarn-lsp Language Server (LSP 3.17) for Claude Code, Neovim, Helix, Zed, and other compatible clients. Ships diagnostics, hover, nested schema-aware completion, document symbols, go-to-definition, references, rename, code lens (run test / run step), formatting, code actions (extract env var, capture this field, scaffold assert from last response), quick fix via tarn::fix_plan, and a JSONPath evaluator (tarn.evaluateJsonpath executeCommand).
  • docs/VSCODE_EXTENSION.md — the VS Code extension in editors/vscode.

A lightweight static docs site now lives in docs/site/index.html and is deployable via GitHub Pages from .github/workflows/docs-site.yml.

Test File Format

Test files use .tarn.yaml and can be organized in any directory structure.

Minimal Test

name: Health check
steps:
  - name: GET /health
    request:
      method: GET
      url: "http://localhost:3000/health"
    assert:
      status: 200

request.method accepts standard verbs and custom tokens such as PURGE or PROPFIND.

Full Format

version: "1"
name: "User CRUD Operations"
description: "Tests complete user lifecycle"
tags: [crud, users, smoke]

env:
  base_url: "http://localhost:3000/api/v1"

defaults:
  headers:
    Content-Type: "application/json"
  timeout: 5000
  retries: 1

tests:
  create_and_verify:
    description: "Create a user, then verify it exists"
    tags: [smoke]
    steps:
      - name: Create user
        request:
          method: POST
          url: "{{ env.base_url }}/users"
          body:
            name: "Jane Doe"
            email: "jane.{{ $random_hex(6) }}@example.com"
        capture:
          user_id: "$.id"
        assert:
          status: 201
          body:
            "$.name": "Jane Doe"
            "$.id": { type: string, not_empty: true }

      - name: Verify user
        request:
          method: GET
          url: "{{ env.base_url }}/users/{{ capture.user_id }}"
        assert:
          status: 200
          body:
            "$.id": "{{ capture.user_id }}"

Setup and Teardown

setup runs once before all tests. teardown runs after all tests even if tests fail.

name: "CRUD with auth"

setup:
  - name: Login
    request:
      method: POST
      url: "{{ env.base_url }}/auth/login"
      body:
        email: "{{ env.admin_email }}"
        password: "{{ env.admin_password }}"
    capture:
      auth_token: "$.token"

teardown:
  - name: Cleanup
    request:
      method: POST
      url: "{{ env.base_url }}/test/cleanup"

tests:
  my_test:
    steps:
      - name: Authenticated request
        request:
          method: GET
          url: "{{ env.base_url }}/users"
          headers:
            Authorization: "Bearer {{ capture.auth_token }}"
        assert:
          status: 200

Assertions

Status

assert:
  status: 200              # exact match
  status: "2xx"            # any 2xx status
  status:                  # set of allowed codes
    in: [200, 201, 204]
  status:                  # range
    gte: 400
    lt: 500

Body (JSONPath)

All body assertions use JSONPath expressions.

Equality:

body:
  "$.name": "Alice"              # string
  "$.age": 30                    # number
  "$.active": true               # boolean
  "$.deletedAt": null            # null
  "$.field": { eq: "value" }     # explicit
  "$.field": { not_eq: "bad" }   # inequality

Numeric comparisons:

body:
  "$.age": { gt: 18, lt: 100 }
  "$.count": { gte: 1, lte: 50 }

String assertions:

body:
  "$.email": { contains: "@example.com" }
  "$.id": { starts_with: "usr_", matches: "^usr_[a-z0-9]+$" }
  "$.name": { not_empty: true }
  "$.notes": { empty: true }
  "$.code": { length: 6 }
  "$.msg": { not_contains: "error" }

Format assertions:

body:
  "$.request_id": { is_uuid: true }
  "$.created_at": { is_date: true }
  "$.client_ip": { is_ipv4: true }
  "$.server_ip": { is_ipv6: true }

Integrity assertions:

body:
  "$": { bytes: 15 }                                # raw response body length
  "$.payload": { sha256: "2cf24dba5fb0a30e..." }    # matched value digest
  "$.legacy": { md5: "5d41402abc4b2a76..." }

Type checks:

body:
  "$.name": { type: string }
  "$.tags": { type: array, length_gt: 0 }
  "$.meta": { type: object }

Existence:

body:
  "$.id": { exists: true }
  "$.internal": { exists: false }

Combined (AND logic):

body:
  "$.id": { type: string, not_empty: true, starts_with: "usr_" }

Headers

assert:
  headers:
    content-type: "application/json"                    # exact match
    content-type: contains "application/json"           # substring
    x-request-id: matches "^[a-f0-9-]{36}$"            # regex

Header names are case-insensitive.

Duration

assert:
  duration: "< 500ms"
  duration: "<= 1s"

Redirects

assert:
  redirect:
    url: "https://api.example.com/health"
    count: 2

redirect.url checks the final response URL after following redirects. redirect.count checks how many redirects were actually followed.

Variables

Environment Variables

Priority Source Example
1 (highest) CLI --var --var base_url=http://staging
2 Shell env ${VAR} password: "${ADMIN_PASSWORD}"
3 tarn.env.local.yaml (gitignored, for secrets)
4 tarn.env.{name}.yaml --env staging loads this
5 tarn.env.yaml default env file
6 (lowest) Inline env: block in the test file itself

Captures (Chaining)

Capture values from responses to use in subsequent steps. Captured values preserve their original JSON types (numbers stay numbers, booleans stay booleans).

steps:
  - name: Login
    request:
      method: POST
      url: "{{ env.base_url }}/auth/login"
      body:
        email: "admin@example.com"
        password: "password123"
    capture:
      token: "$.token"              # JSONPath capture from body
      user_id: "$.user.id"          # nested path

  - name: Use token
    request:
      method: GET
      url: "{{ env.base_url }}/users"
      headers:
        Authorization: "Bearer {{ capture.token }}"

Header capture — capture values from response headers with optional regex:

capture:
  session_token:
    header: "set-cookie"
    regex: "session_token=([^;]+)"
  request_id:
    header: "x-request-id"

Cookie capture — capture a response cookie by name from Set-Cookie:

capture:
  session_cookie:
    cookie: "session"

Status capture — capture the HTTP status code as a number:

capture:
  status_code:
    status: true

Final URL capture — capture the final response URL after redirects:

capture:
  final_url:
    url: true

JSONPath with regex — extract a sub-match from a body field:

capture:
  user_id:
    jsonpath: "$.message"
    regex: "ID: (\\w+)"

Whole-body regex — extract from the full response body string:

capture:
  body_word:
    body: true
    regex: "plain (text)"

Transform-lite in interpolation — reshape captured arrays and collections without dropping into Lua:

request:
  form:
    first_tag: "{{ capture.tags | first }}"
    last_tag: "{{ capture.tags | last }}"
    tag_count: "{{ capture.tags | count }}"
    joined_tags: "{{ capture.tags | join('|') }}"
    words: "{{ capture.message | split(' ') | count }}"
    normalized: "{{ capture.message | replace(' response', '') }}"
    status_code: "{{ capture.status_text | to_int }}"
    payload: "{{ capture.user | to_string }}"

first and last expect arrays. count works on arrays, objects, and strings. join(...) joins array items after converting each item to its string form. split(...) and replace(..., ...) operate on strings. to_int parses integer strings, and to_string stringifies any captured value.

Built-in Functions

"{{ $uuid }}"                    # UUID v4
"{{ $random_hex(8) }}"           # 8-char hex string
"{{ $random_int(1, 100) }}"      # random integer in range
"{{ $timestamp }}"               # unix timestamp
"{{ $now_iso }}"                 # ISO 8601 datetime

Cookies

Tarn automatically captures Set-Cookie headers and sends stored cookies on subsequent requests. This is enabled by default.

name: Auth flow
steps:
  - name: Login
    request:
      method: POST
      url: "{{ env.base_url }}/auth/login"
      body:
        email: "admin@test.com"
        password: "secret"
    # Set-Cookie from response is automatically stored
    assert:
      status: 200

  - name: Access protected resource
    request:
      method: GET
      url: "{{ env.base_url }}/profile"
    # Cookie header is automatically sent
    assert:
      status: 200

Disable automatic cookies per file:

cookies: "off"

Or reset the default jar between named tests in a file so IDE subset runs and flaky suites never see session state from a prior test. Setup and teardown still share the file-level jar. Named jars (multi-user scenarios) are untouched.

cookies: "per-test"

The --cookie-jar-per-test CLI flag forces per-test isolation regardless of the file's declared mode (except when the file sets cookies: "off", which always wins).

Auth

Tarn supports first-class bearer and basic auth helpers, while keeping explicit Authorization headers as the escape hatch:

request:
  auth:
    bearer: "{{ env.token }}"
  headers:
    X-API-Key: "{{ env.api_key }}"

Basic auth:

request:
  auth:
    basic:
      username: "{{ env.username }}"
      password: "{{ env.password }}"

You can also set defaults.auth once per file. If headers.Authorization is already present, Tarn leaves it unchanged.

Step-Level Cookie Control

Use cookies: false on a step to bypass the cookie jar entirely. No cookies are sent and no Set-Cookie headers are captured:

steps:
  - name: Login
    request:
      method: POST
      url: "{{ env.base_url }}/auth/login"
      body:
        email: "admin@test.com"
        password: "secret"
    assert:
      status: 200

  - name: Test unauthenticated access
    cookies: false
    request:
      method: GET
      url: "{{ env.base_url }}/profile"
    assert:
      status: 401

Named Cookie Jars

For multi-user scenarios, use named jars to maintain separate cookie sessions:

steps:
  - name: Login as admin
    cookies: "admin"
    request:
      method: POST
      url: "{{ env.base_url }}/auth/login"
      body:
        email: "admin@test.com"
        password: "secret"

  - name: Login as viewer
    cookies: "viewer"
    request:
      method: POST
      url: "{{ env.base_url }}/auth/login"
      body:
        email: "viewer@test.com"
        password: "secret"

  - name: Admin can manage users
    cookies: "admin"
    request:
      method: GET
      url: "{{ env.base_url }}/admin/users"
    assert:
      status: 200

  - name: Viewer cannot manage users
    cookies: "viewer"
    request:
      method: GET
      url: "{{ env.base_url }}/admin/users"
    assert:
      status: 403

Each named jar is independent — cookies captured in "admin" are never sent with "viewer" requests. Steps without a cookies: field (or with cookies: true) use the default jar.

Persist Cookie Jars

Use tarn run --cookie-jar .tarn-cookies.json to preload jars from disk and write back the updated state after the run. The file stores named jars too, so multi-user sessions can survive across runs.

--cookie-jar currently works only with sequential execution. Combine it with --parallel only after jar sharing becomes deterministic.

CSRF Protection

When the cookie jar sends cookies automatically, frameworks with CSRF protection (e.g., Better Auth) may reject requests that lack an Origin header. Add it to defaults to fix:

defaults:
  headers:
    Content-Type: "application/json"
    Origin: "http://localhost:3000"

If your app derives the expected origin from the request URL, set Origin to match env.base_url:

defaults:
  headers:
    Origin: "{{ env.base_url }}"

Form URL-Encoding

Send application/x-www-form-urlencoded payloads with form::

steps:
  - name: Login form
    request:
      method: POST
      url: "{{ env.base_url }}/auth/login"
      form:
        email: "user@example.com"
        password: "{{ env.password }}"
    assert:
      status: 200

Tarn URL-encodes the fields and auto-sets Content-Type: application/x-www-form-urlencoded unless you override it explicitly.

Note: form cannot be combined with body, graphql, or multipart on the same step.

Multipart / File Upload

Send multipart form data for file uploads using the multipart: field:

steps:
  - name: Upload photo
    request:
      method: POST
      url: "{{ env.base_url }}/api/photos"
      headers:
        Authorization: "Bearer {{ capture.token }}"
      multipart:
        fields:
          - name: "title"
            value: "My Photo"
          - name: "description"
            value: "A test upload"
        files:
          - name: "photo"
            path: "./fixtures/test.jpg"
            content_type: "image/jpeg"
          - name: "thumbnail"
            path: "./fixtures/thumb.png"
            filename: "custom-name.png"
    assert:
      status: 201

Note: multipart cannot be combined with body, form, or graphql on the same step.

Includes

Reuse shared step sequences across test files with include: directives:

name: User tests
setup:
  - include: ./shared/auth-setup.tarn.yaml
steps:
  - name: Get users
    request:
      method: GET
      url: "{{ env.base_url }}/users"
    assert:
      status: 200

The included file's setup and steps are inlined at the include point. Includes work in setup, teardown, steps, and tests.*.steps. Circular includes are detected and rejected.

Includes also support lightweight parametrization and deep overrides for reusable step packs:

steps:
  - include: ./shared/user-pack.tarn.yaml
    with:
      tenant: "acme"
      user_id: 42
    override:
      request:
        headers:
          X-Tenant: "acme"

Inside the included file, use {{ params.tenant }} and {{ params.user_id }} placeholders.

GraphQL

Native GraphQL support with the graphql: block. Automatically sets Content-Type: application/json and constructs the standard GraphQL JSON body.

steps:
  - name: Get user
    request:
      method: POST
      url: "{{ env.base_url }}/graphql"
      graphql:
        query: |
          query GetUser($id: ID!) {
            user(id: $id) {
              id
              name
              email
            }
          }
        variables:
          id: "{{ capture.user_id }}"
        operation_name: "GetUser"
    assert:
      status: 200
      body:
        "$.data.user.name": "Alice"
        "$.errors": { exists: false }

Polling

Re-execute a step until a condition is met. Useful for async workflows where you need to wait for a state change.

steps:
  - name: Create export
    request:
      method: POST
      url: "{{ env.base_url }}/exports"
    capture:
      export_id: "$.id"
    assert:
      status: 202

  - name: Wait for completion
    request:
      method: GET
      url: "{{ env.base_url }}/exports/{{ capture.export_id }}"
    poll:
      until:
        body:
          "$.status": "completed"
      interval: "2s"
      max_attempts: 15
    assert:
      status: 200
      body:
        "$.status": "completed"

poll.until uses the same assertion syntax. The step re-executes every interval until the until condition passes or max_attempts is reached.

Lua Scripting

For logic that goes beyond declarative assertions, use inline Lua scripts. Scripts run after the HTTP response is received and have access to response and captures.

steps:
  - name: Validate complex logic
    request:
      method: GET
      url: "{{ env.base_url }}/users"
    script: |
      -- Access response
      assert(response.status == 200, "Expected 200")

      -- Work with the response body (Lua table)
      local users = response.body.users
      assert(#users > 0, "Expected at least one user")

      -- Cross-field validation
      for _, user in ipairs(users) do
        assert(user.email:find("@"), "Invalid email for " .. user.name)
      end

      -- Set captures for subsequent steps
      captures.first_user_id = users[1].id
    assert:
      status: 200

Available in Lua:

  • response.status — HTTP status code
  • response.headers — response headers table
  • response.body — response body as Lua table (auto-parsed from JSON)
  • captures — read/write captures table
  • assert(condition, message) — assertion (collected, not thrown)
  • json.decode(string) — parse a JSON string into a Lua table
  • json.encode(value) — serialize a Lua value to a JSON string

CLI Reference

tarn run [PATH] [OPTIONS]          Run test files
tarn bench <PATH> [OPTIONS]        Benchmark a step
tarn validate [PATH] [--format]    Validate YAML (--format human|json)
tarn fmt [PATH] [--check]          Normalize Tarn YAML
tarn list                          List all tests
tarn import-hurl <PATH>            Convert common-case Hurl files to Tarn
tarn init                          Scaffold a new project
tarn update                        Update to the latest version
tarn update --check                Check for updates without installing
tarn completions <SHELL>           Generate shell completions

tarn run Options

Flag Description
--format <FORMAT> Repeatable. Supports human, json, junit, tap, html, curl, curl-all, or FORMAT=PATH
--json-mode <MODE> For JSON outputs: verbose (default) or compact
--tag <TAGS> Filter by tag (comma-separated, AND logic)
--select <FILE[::TEST[::STEP]]> Narrow execution to specific files, tests, or steps (repeatable; ANDs with --tag)
--var <KEY=VALUE> Override env variables (repeatable)
--env <NAME> Load tarn.env.{name}.yaml
-v, --verbose Print full request/response for every step
--only-failed Show only failed tests and steps (summary counts stay accurate)
--no-progress Disable streaming progress output; print the final report in one batch
--ndjson Stream machine-readable NDJSON events to stdout (for editor integrations, MCP, structured CI)
--dry-run Show interpolated requests without sending
-w, --watch Re-run on file changes
--parallel Run test files in parallel
-j, --jobs <N> Number of parallel workers (default: CPU count)

Examples

tarn run                                        # all tests
tarn run tests/auth.tarn.yaml                   # specific file
tarn run --tag smoke                            # filter by tag
tarn run --env staging                          # staging env
tarn run --var base_url=http://localhost:8080    # override var
tarn run --format json                          # JSON for LLM/CI
tarn run --format json --json-mode compact     # smaller JSON for automation loops
tarn run --format html                          # HTML dashboard
tarn run --format curl                          # failed requests as curl
tarn run --format curl-all=reports/replay.sh    # full suite replay script
tarn run --format human --format json=reports/run.json --format junit=reports/junit.xml
tarn run --format human,json=reports/run.json   # comma-separated also works
tarn run --watch                                # re-run on changes
tarn run --parallel --jobs 4                    # parallel execution
tarn run -v                                     # verbose
tarn run --dry-run                              # preview only
tarn run --only-failed                          # hide passing tests, show failures only
tarn run --no-progress                          # disable streaming, batch dump at end
tarn run --only-failed --format json            # CI-friendly: only failed items in JSON
tarn run --select tests/users.tarn.yaml::login   # run just the "login" test in one file
tarn run --select tests/users.tarn.yaml::login::2   # run just step index 2 of login
tarn run --select "a.tarn.yaml::login" --select "b.tarn.yaml::checkout"  # union across files
tarn fmt tests/                                  # rewrite a directory in place
tarn fmt tests/auth.tarn.yaml --check            # CI-style formatting check

Structured Validation (tarn validate --format json)

tarn validate --format json emits a machine-readable report so editors and CI can surface parse errors inline. The schema:

{
  "files": [
    {
      "file": "tests/users.tarn.yaml",
      "valid": false,
      "errors": [
        { "message": "found unexpected end of stream ...", "line": 14, "column": 7 }
      ]
    }
  ]
}
  • line and column are populated for YAML syntax errors (derived from serde_yaml's error location).
  • Parser semantic errors (unknown fields, shape mismatches) surface message only when the underlying error does not carry a location.
  • Exit code is 0 when every file is valid, 2 otherwise. The human format (--format human, the default) is unchanged.

Environment Discovery (tarn env --json)

tarn env --json prints the project's named environments in a stable schema so editors can populate pickers and previews:

{
  "project_root": "/path/to/project",
  "default_env_file": "tarn.env.yaml",
  "environments": [
    {
      "name": "staging",
      "source_file": "tarn.env.staging.yaml",
      "vars": {
        "base_url": "https://staging.example.com",
        "api_token": "***"
      }
    }
  ]
}

Inline vars from tarn.config.yaml are redacted when the key matches redaction.env (case-insensitive), so tarn env --json never prints literal secrets. Environments are sorted alphabetically by name.

Streaming Progress

By default tarn run streams per-test output as each test finishes instead of dumping everything at the end. The behaviour adapts to how stdout is used:

  • Sequential (default) — each test is printed the moment it completes. You see progress live as the suite runs.
  • Parallel (--parallel) — each file is printed atomically when it completes, so output from concurrently running files never interleaves.
  • Stdout is human — streaming writes directly to stdout and the final emit prints only the summary line (no duplication).
  • Stdout is a structured format (json, junit, tap, html, curl) — progress streams to stderr so stdout stays pure and parseable.

Pass --no-progress to disable streaming entirely and restore the old "batch at end" behaviour (useful for CI logs that already capture per-line timestamps).

NDJSON Streaming (--ndjson)

tarn run --ndjson streams machine-readable events to stdout, one JSON object per line. Designed for editor integrations (live Test Explorer updates), MCP clients, and CI pipelines that want structured progress without post-processing the final report.

Event types, in order:

  • file_started — a test file has begun running
  • step_finished — one step finished (with phase: "setup" | "test" | "teardown"). On failure, also carries failure_category, error_code, and assertion_failures[]
  • test_finished — a named test finished, with per-step counts
  • file_finished — a file finished, with its own summary
  • done — emitted once at the very end, carrying the aggregated summary for the whole run

--ndjson composes with file-bound --format targets, so you can stream live progress and write a final report at the same time:

# Stream NDJSON to stdout, final JSON report to disk
tarn run --ndjson --format json=reports/run.json | jq '.event'

# Pure NDJSON (default human output is silently dropped on stdout)
tarn run --ndjson

--ndjson collides with any other structured format writing to stdout (e.g. --format json). Route the other format to a file, or pick one of the two streams.

In parallel mode (--parallel), each file's event stream is emitted atomically on file_finished so events from concurrently running files never interleave.

--only-failed works with both streaming and batch modes: passing tests and steps are omitted everywhere, but the final summary still reports total passed/failed counts.

Exit Codes

Code Meaning
0 All tests passed
1 One or more tests failed
2 Configuration/parse error
3 Runtime error (network, timeout, script)

Output Formats

You can emit multiple formats in one run. Keep at most one bare non-HTML format for stdout and send the rest to files:

tarn run \
  --format human \
  --format json=reports/run.json \
  --format junit=reports/junit.xml \
  --format html=reports/run.html \
  --format curl=reports/failures.sh \
  --format curl-all=reports/replay.sh

JSON (--format json)

Structured JSON with versioned schema. Key design:

  • schema_version: 1 for forward compatibility
  • Full request/response included only for failed steps
  • failure_category on failures: assertion_failed, connection_error, timeout, parse_error, capture_error, unresolved_template
  • Stable error_code and remediation_hints are included on failed steps for automation-friendly diagnostics
  • response_status and response_summary on all executed steps (passed and failed) — AI agents can see what a passed step returned
  • captures_set on steps listing which capture variables were set; captures map on test groups showing all resolved values
  • --json-mode compact keeps the same top-level schema but drops passed assertion details and truncates response bodies to ~200 chars
  • Sensitive headers are redacted by default and can be customized per file with top-level redaction:
  • request is present for failed executed steps; response is omitted for connection/setup failures where no response exists

Schema files:

  • test files: schemas/v1/testfile.json
  • JSON report output: schemas/v1/report.json
{
  "schema_version": 1,
  "summary": { "status": "FAILED", "steps": { "total": 5, "passed": 4, "failed": 1 } },
  "files": [{
    "tests": [{
      "captures": { "user_id": "usr_123", "token": "abc" },
      "steps": [{
        "name": "Create user",
        "status": "PASSED",
        "response_status": 201,
        "response_summary": "201 Created: Object{3 keys}",
        "captures_set": ["user_id"]
      }, {
        "name": "Update user",
        "status": "FAILED",
        "response_status": 400,
        "response_summary": "400 Bad Request: name required",
        "failure_category": "assertion_failed",
        "error_code": "assertion_mismatch",
        "remediation_hints": ["..."],
        "assertions": {
          "failures": [{ "assertion": "status", "expected": "201", "actual": "400", "message": "..." }]
        },
        "request": { "method": "POST", "url": "..." },
        "response": { "status": 400, "body": { "error": "name required" } }
      }]
    }]
  }]
}

Curl (--format curl, --format curl-all)

curl exports only failed executed requests. curl-all exports every executed request in run order, including setup and teardown.

tarn run --format human --format curl=reports/failures.sh
tarn run --format curl-all=reports/replay.sh

Also supports: Human (colored terminal), JUnit XML, TAP, HTML (self-contained dashboard).

Example:

redaction:
  headers:
    - authorization
    - x-session-token
  env:
    - api_token
  captures:
    - session_token
  replacement: "[redacted]"

Performance Testing

Reuses your existing test files for benchmarking.

tarn bench tests/health.tarn.yaml -n 1000 -c 50
tarn bench tests/health.tarn.yaml -n 500 -c 25 --ramp-up 5s
tarn bench tests/health.tarn.yaml --format json     # for CI thresholds
tarn bench tests/health.tarn.yaml --format csv --export json=reports/bench.json
tarn bench tests/health.tarn.yaml --fail-under-rps 200 --fail-above-p95-ms 80

 TARN BENCH  GET http://localhost:3000/health — 200 requests, 20 concurrent

  Requests:      200 total, 200 ok, 0 failed (0.0%)
  Throughput:    3125.0 req/s

  Latency:
    min        1ms
    p50        2ms
    p95        43ms
    p99        45ms
    max        45ms

MCP Server

Tarn includes an MCP (Model Context Protocol) server for direct integration with AI coding tools.

Setup

The simplest approach is a project-level .mcp.json in the repo root (works with Claude Code and other MCP-compatible tools):

{
  "mcpServers": {
    "tarn": {
      "command": "tarn-mcp",
      "args": []
    }
  }
}

Alternatively, add to your Claude Code project settings (.claude/settings.json):

{
  "mcpServers": {
    "tarn": {
      "command": "tarn-mcp",
      "args": []
    }
  }
}

For Cursor, add to .cursor/mcp.json:

{
  "mcpServers": {
    "tarn": {
      "command": "tarn-mcp"
    }
  }
}

Available Tools

Tool Description
tarn_run Run tests, returns structured JSON results
tarn_validate Validate YAML syntax without executing
tarn_list List all tests and their steps
tarn_fix_plan Analyze a Tarn JSON report and return prioritized next actions

The MCP server lets your AI agent write .tarn.yaml tests, execute them, parse structured results, and iterate — all without leaving the editor.

Typical agent loop:

  1. tarn_list to discover tests and steps
  2. tarn_validate after generating YAML
  3. tarn_run to get structured failures
  4. tarn_fix_plan to turn the latest report into machine-friendly next steps
  5. inspect failure_category, error_code, assertions.failures, and optional request/response
  6. patch the test or application code
  7. rerun until summary status is PASSED

See docs/MCP_WORKFLOW.md, docs/AI_WORKFLOW_DEMO.md, and docs/CONFORMANCE.md.

Claude Code Plugin

Tarn ships two Claude Code plugins from a single marketplace. They solve different problems and can be installed independently or together:

  1. tarn (top-level plugin/) — bundles the tarn-mcp MCP server and the tarn-api-testing skill. Gives your agent structured API testing capabilities: tarn_run, tarn_validate, tarn_list, tarn_fix_plan.
  2. tarn-lsp (editors/claude-code/tarn-lsp-plugin/) — registers the tarn-lsp language server with Claude Code's LSP plugin system so you get full .tarn.yaml language intelligence (diagnostics, hover, completion, code lens, code actions, quick fix, rename, go-to-definition, and the JSONPath evaluator) while editing in Claude Code.

Both plugins live in the same marketplace (the repo root .claude-plugin/marketplace.json). Register it once, then install either or both:

# 1. Register the marketplace (once)
claude plugin marketplace add NazarKalytiuk/hive

# 2a. Install the MCP + skill plugin
claude plugin install tarn@tarn

# 2b. Install the LSP plugin (project scope — see caveat below)
claude plugin install tarn-lsp@tarn --scope project

After installing tarn, Claude Code can write, run, and debug .tarn.yaml tests directly via the bundled MCP server and skill. See MCP Server and Claude Code Skill for what each component provides.

tarn — MCP + skill plugin

Manual setup

If you prefer manual configuration, add the MCP server to a project-level .mcp.json in the repo root:

{
  "mcpServers": {
    "tarn": {
      "command": "tarn-mcp",
      "args": []
    }
  }
}

This is equivalent to configuring the MCP server in .claude/settings.json but is portable across editors and tools that support MCP.

Plugin metadata

The tarn plugin configuration lives in .claude-plugin/:

  • plugin.json — name, version, description, author, and repository URL
  • marketplace.json — marketplace listing with owner info and the plugin registry (both tarn and tarn-lsp)

tarn-lsp — language server plugin

Separate install from the MCP plugin above (same marketplace though). This one registers tarn-lsp for .tarn.yaml / .yaml / .yml via Claude Code's LSP plugin system so every feature documented in docs/TARN_LSP.md is available while you edit in Claude Code.

Prerequisites:

  • Claude Code 2.0.74+
  • tarn-lsp binary available on $PATH — install with cargo install --path tarn-lsp from this repo, or symlink a workspace build (ln -s $(pwd)/target/release/tarn-lsp /usr/local/bin/tarn-lsp)

Install (from inside a Claude Code session):

/plugin marketplace add NazarKalytiuk/hive
/plugin install tarn-lsp@tarn --scope project
/reload-plugins

Already registered the marketplace for the tarn plugin? Skip the add line — both plugins share the same marketplace now. Substitute /absolute/path/to/repo for NazarKalytiuk/hive if you want to install from a local checkout instead.

Compound-extension caveat: Claude Code's LSP plugin system only supports simple file extensions, so the tarn-lsp plugin claims all .yaml and .yml files in any project it is installed in (not just .tarn.yaml). Always install with --scope project in Tarn-focused repos only — do not install it globally if you also edit unrelated YAML in Claude Code.

See editors/claude-code/tarn-lsp-plugin/README.md for the full spec, troubleshooting, and the list of supported LSP features.

opencode

opencode supports Tarn through config only — there is no plugin installer or marketplace, so integration is three files checked into your repo:

your-repo/
├── opencode.jsonc                          # MCP + LSP registration
└── .opencode/skills/tarn-api-testing/      # agent-visible skill
    └── SKILL.md

This repo ships exactly this layout at opencode.jsonc and .opencode/skills/tarn-api-testing/ (the skill is a symlink to the canonical plugin/skills/tarn-api-testing/). Clone the repo, install tarn-mcp and tarn-lsp on $PATH, run opencode inside — MCP tools, .tarn.yaml diagnostics/hover/completion, and the tarn-api-testing skill light up immediately.

To mirror the setup in your own repo, copy the snippet from editors/opencode/opencode.example.jsonc into your own opencode.jsonc:

{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "tarn": { "type": "local", "command": ["tarn-mcp"], "enabled": true }
  },
  "lsp": {
    "tarn": { "command": ["tarn-lsp"], "extensions": [".yaml", ".yml"] }
  }
}

Compound-extension caveat: opencode's LSP matcher uses path.parse(file).ext, so the tarn LSP entry claims every .yaml / .yml file in the workspace (not just .tarn.yaml) — the same limitation as Claude Code. Keep this in project-level opencode.jsonc, not your global config.

See editors/opencode/README.md for prerequisites, troubleshooting, and the full skill-install flow.

Claude Code Skill

The skills/tarn-api-testing/ directory contains a Claude Code skill that teaches AI agents how to write, run, debug, and iterate on Tarn tests. The skill is automatically loaded when an agent encounters API testing tasks.

What the skill provides:

  • Core workflow (write → validate → run → inspect → fix → rerun)
  • Complete command reference with all CLI flags
  • Test file format with minimal and full-featured examples
  • Environment variable resolution chain
  • Capture formats (JSONPath, headers, cookies, URL, status, body)
  • Assertion operator quick reference
  • JSON output schema and failure category taxonomy
  • Diagnosis loop for structured failure triage
  • MCP integration setup for Claude Code, opencode, Cursor, and Windsurf

Reference docs in skills/tarn-api-testing/references/:

File Contents
yaml-format.md Complete .tarn.yaml schema with all properties
assertion-reference.md Every assertion operator with examples
json-output.md Structured JSON report schema and diagnosis algorithm
mcp-integration.md MCP server setup and tool reference

The skill triggers on keywords like "API test", "tarn", ".tarn.yaml", "test this endpoint", "smoke test", and "integration test for API".

Troubleshooting

Common cases:

  • connection_error: server is down, wrong host/port, DNS issue, TLS/connect failure
  • timeout: step timed out before receiving a complete response
  • assertion_failed: request succeeded, but status/header/body/duration check failed
  • capture_error: the step passed assertions, but extraction failed afterward
  • parse_error: invalid YAML, invalid JSONPath, or invalid config surface

Agent diagnosis loop:

  1. run tarn validate first for syntax/config errors
  2. run tarn run --format json
  3. read failure_category before reading the message text
  4. if response exists, inspect it before editing assertions
  5. if request.url still contains {{ ... }}, fix env/capture interpolation before retrying

Non-JSON bodies:

  • Tarn preserves plain text / HTML responses as JSON strings in the structured report
  • use body: { "$": "plain text response" } to assert the whole root string when needed

Intentional Gaps

Tarn does not aim for full Hurl parity. The main intentionally unclosed gaps are:

  • XPath / HTML assertions and captures
  • full Hurl-style filter DSL
  • exotic auth and libcurl-specific transport features
  • OpenAPI-first generation workflows

GitHub Action

- uses: NazarKalytiuk/hive@v1
  with:
    path: tests/
    format: junit
    env: staging

Inputs:

Input Default Description
path tests Test file or directory
format human Output format
env — Environment name
tag — Tag filter
version latest Tarn version
vars — Variables (newline-separated KEY=VALUE)

Configuration

tarn.config.yaml (optional)

test_dir: "tests"
env_file: "tarn.env.yaml"
timeout: 10000
retries: 0
parallel: false
defaults:
  connect_timeout: 1000
  follow_redirects: true
redaction:
  headers: ["authorization", "cookie"]
environments:
  staging:
    env_file: "env/staging.yaml"
    vars:
      base_url: "https://staging.example.com"

Behavior:

  • test_dir sets the default discovery directory for tarn run, tarn validate, and tarn list
  • env_file changes the root env file name; Tarn also checks .{name} and .local variants
  • defaults acts as project-wide request policy for headers/auth/timeouts/retries/redirects/delay
  • redaction provides a project-wide default report sanitization policy
  • environments makes named --env profiles first-class and powers tarn env
  • parallel: true makes parallel file execution the default for tarn run

File-level defaults

defaults:
  headers:
    Content-Type: "application/json"
  timeout: 5000
  retries: 1
  delay: "100ms"    # default delay before each request

Step Options

Retries

retries: 3    # retry up to 3 times on failure (exponential backoff)

Timeout

timeout: 30000    # 30 seconds for this step

Delay

delay: "2s"    # wait before executing

JSON Schema

Add to the top of your .tarn.yaml files for IDE autocompletion:

# yaml-language-server: $schema=https://raw.githubusercontent.com/NazarKalytiuk/hive/main/schemas/v1/testfile.json
name: My test
steps: ...

The schema is bundled at schemas/v1/testfile.json in the repository. The structured report schema is bundled at schemas/v1/report.json.

VS Code Extension

A full-featured Tarn extension lives in editors/vscode and is published from tagged releases to both the VS Marketplace (nazarkalytiuk.tarn-vscode) and Open VSX via .github/workflows/vscode-extension-release.yml. Current version: 0.6.1.

Running tests from the editor

  • Test Explorer discovery — .tarn.yaml files are indexed into a file → test → step tree. Run and Dry Run profiles, cancellable runs, and live streaming via tarn run --ndjson keep the UI in sync with long runs.
  • CodeLens above every test and step for Run, Dry Run, and Run step — no Test Explorer navigation required.
  • Rich failure peek view — on failure you get a unified diff of expected vs actual, plus the full request, response, remediation hints, failure_category, and error_code pulled straight from Tarn's JSON report.
  • Tag filter command and an "Install / Update Tarn" helper command, both surfaced in the command palette.
  • Output channel streams tarn stdout/stderr for each run.

Environment management

  • Environment picker with a status-bar entry, persisted per workspace.
  • tarn.defaultEnvironment setting for the initial pick.
  • Status bar entries summarize active environment, tag filter, and last-run status.

Language features

  • Tarn file association for *.tarn.yaml and *.tarn.yml.
  • Full JSON schema validation for both test files and tarn-report.json via the redhat.vscode-yaml extension dependency.
  • Snippet library for test skeletons, polling, multipart, GraphQL, form requests, and includes. Prefix and coverage details live in editors/vscode/README.md.
  • Experimental LSP client (off by default) — the window-scoped tarn.experimentalLspClient setting spawns the tarn-lsp server alongside the extension's in-process providers. This is Phase V scaffolding; no feature has migrated to the LSP path yet, so leave it disabled unless you are testing the handoff.

Workspace trust and remote development

  • Trusted / Untrusted workspace aware. In untrusted workspaces, Tarn features run in a read-only mode — discovery and schema validation still work, but commands that would execute tarn are disabled.
  • Remote Development audited end-to-end: Dev Container, GitHub Codespaces, WSL, and Remote SSH all work without additional configuration. See docs/VSCODE_REMOTE.md.

Public API

The extension exports a TarnExtensionApi for other extensions to consume:

const tarn = vscode.extensions
  .getExtension('nazarkalytiuk.tarn-vscode')
  ?.exports as TarnExtensionApi | undefined;

See editors/vscode/docs/API.md for the surface and stability guarantees.

Reference

Shell Completions

tarn completions bash > /etc/bash_completion.d/tarn
tarn completions zsh > ~/.zsh/completions/_tarn
tarn completions fish > ~/.config/fish/completions/tarn.fish

Development

git clone https://github.com/NazarKalytiuk/hive.git
cd tarn

cargo build                    # build
cargo test --all               # test suite
cargo clippy                   # lint
cargo fmt                      # format
bash scripts/ci/smoke.sh       # release-path smoke test

# Run demo server + examples
PORT=3333 cargo run -p demo-server &
cargo run -p tarn -- run examples/ --var base_url=http://localhost:3333

See docs/RELEASE_VERIFICATION.md for the broader release-candidate checklist, including watch-mode and installer verification.

Architecture

Pipeline: parse YAML → resolve env → interpolate → execute HTTP → assert → report

Module Role
model.rs Serde structs for .tarn.yaml
parser.rs YAML loading + validation
env.rs 6-layer env resolution
interpolation.rs {{ }} template engine
runner.rs Orchestrator (setup → tests → teardown)
http.rs HTTP client (reqwest)
capture.rs JSONPath + header extraction
cookie.rs Automatic cookie jar
config.rs tarn.config.yaml parsing
builtin.rs Built-in functions ($uuid, $timestamp, etc.)
update.rs Self-update mechanism
assert/ Status, body, headers, duration
report/ Human, JSON, JUnit, TAP, HTML
scripting.rs Lua scripting engine (mlua)
watch.rs File watcher (notify)
bench.rs Performance testing (async)

License

MIT