checkmate-cli 0.4.1

# Frequently Asked Questions

## General

### What is Checkmate?

Checkmate is an AI-native API testing framework. It uses:
- **YAML** for test specifications
- **Clove** for JSON assertions
- **Git** for history tracking

### Why YAML for test specs?

- Human-readable and editable
- Easy for AI agents to generate
- Declarative, not procedural
- Widely understood format

### Why Clove instead of JSONPath?

Clove is designed specifically for Checkmate:
- Simpler syntax for common operations
- Built-in `$[@prev]` for multi-request tests
- Method chaining (`.length()`, `.upper()`)
- Better error messages

### How does Checkmate compare to Postman?

| Feature | Checkmate | Postman |
|---------|-----------|---------|
| Format | YAML (text) | JSON (GUI) |
| Version Control | Git-native | Export/Import |
| AI Integration | Built-in | Limited |
| CI/CD | Command line | Newman |
| Learning Curve | Low | Medium |

Checkmate is optimized for CLI and AI workflows.

### How does it compare to curl scripts?

Checkmate provides:
- Structured assertions
- Test history with git context
- Configuration management
- Reusable request templates

Curl scripts require manual response checking.

## Usage

### Can I use environment variables in specs?

Currently, use shell expansion:

```bash
TOKEN="abc123" envsubst < template.yaml > .checkmate/tests/test.yaml
```

Or use environment variable overrides:

```bash
CM_ENV__BASE_URL="$API_URL" cm test run
```

### Can I run tests in parallel?

Currently tests run sequentially. Parallel execution is planned.

### How do I test authenticated endpoints?

Include auth headers in requests:

```yaml
requests:
  authenticated:
    headers:
      Authorization: "Bearer your-token"
```

### Can I test GraphQL APIs?

Yes, POST to your GraphQL endpoint:

```yaml
requests:
  graphql_query:
    body:
      query: |
        query {
          users {
            id
            name
          }
        }

tests:
  get_users:
    endpoint: /graphql
    method: POST
    requests: [graphql_query]
    assertions:
      - query: "$[data][users]"
        expect_type: array
```

### Can I test WebSocket APIs?

Not currently. Checkmate focuses on HTTP/REST APIs.

### How do I test file uploads?

Multipart form data is not yet supported. Use curl for file uploads.

## Configuration

### Where is configuration stored?

Priority (highest first):
1. Environment variables (`CM_*`)
2. Project config (`.checkmate/config.toml`)
3. User config (`~/.config/checkmate/config.toml`)
4. Built-in defaults

### Can I have different configs for different environments?

Yes, use environment variables:

```bash
# Development
cm test run

# Staging
CM_ENV__BASE_URL="https://staging.api.com" cm test run

# Production
CM_ENV__BASE_URL="https://api.com" cm test run
```

### What's stealth mode?

Stealth mode keeps `.checkmate/` local (not tracked in git):

```bash
cm init --url http://localhost:8080 --stealth
```

Useful for personal testing on shared repositories.

## History & Git

### How is history tracked?

Each test run creates a record in `.checkmate/runs/runs.jsonl` with:
- Run ID and timestamp
- Test results summary
- Git context (commit, branch, dirty status)

### Is history committed to git?

By default, no. `.checkmate/runs/.gitignore` excludes JSONL files.

To track history:
```bash
rm .checkmate/runs/.gitignore
git add .checkmate/runs/
```

### Can I clear history?

```bash
rm .checkmate/runs/runs.jsonl
```

### How do I find when tests started failing?

```bash
cm history -n 50 | grep "✗"
```

Then inspect the commit:
```bash
cm show cm-run-xyz
```

## CI/CD

### How do I use Checkmate in CI?

```yaml
# GitHub Actions
steps:
  - uses: actions/checkout@v3
  - name: Install Checkmate
    run: |
      cargo install --path crates/checkmate-cli
  - name: Run tests
    env:
      CM_ENV__BASE_URL: ${{ secrets.API_URL }}
    run: cm test run
```

### How do I fail the build on test failures?

Checkmate exits with code 1 on failures:

```bash
cm test run || exit 1
```

### Can I get JUnit XML output?

Not currently. JSON output can be converted:

```bash
cm test run > results.json
# Use jq or script to convert to JUnit XML
```

## Hooks

### When do hooks run?

| Hook | When |
|------|------|
| `pre_run` | Before tests start |
| `post_run` | After tests (always) |
| `on_pass` | All tests passed |
| `on_fail` | Any tests failed |

### Are hooks blocking?

No, hooks run asynchronously (fire-and-forget).

### Can hooks fail the test run?

No, hook failures are warnings only.

## AI Integration

### How do I use Checkmate with Claude?

Install the Claude plugin:
```bash
cp -r claude-plugin ~/.claude/plugins/checkmate
```

The plugin injects test context automatically.

### Can AI agents create test specs?

Yes, that's a primary use case. Ask Claude:

> "Create a test spec for the /api/users endpoint"

### How do I onboard an AI agent?

```bash
cm onboard
```

Or add to AGENTS.md:
```markdown
## API Testing
Use `cm test run` for API tests.
See `cm docs` for documentation.
```

## Troubleshooting

### See [TROUBLESHOOTING.md](TROUBLESHOOTING.md)

Common issues:
- "No .checkmate/ found" → Run `cm init`
- Connection refused → Check `cm config show`
- Tests not found → Check `.checkmate/tests/`

## Feature Requests

### Is X feature planned?

Check the issue tracker. Common requests:
- Parallel test execution
- JUnit XML output
- Environment variable interpolation
- gRPC support

### How do I request a feature?

Open an issue on GitHub with:
- Use case description
- Expected behavior
- Example spec if applicable

---

## See Also

- [Quick Start](QUICKSTART.md)
- [Troubleshooting](TROUBLESHOOTING.md)
- [CLI Reference](CLI_REFERENCE.md)