checkmate-cli 0.4.1

# AI Agent Integration

Guide for AI agents working with Checkmate.

## Quick Reference

```bash
# Initialize (provide URL explicitly)
cm init --url http://localhost:8080

# Run all tests
cm test run

# Run specific spec
cm test run users

# List tests
cm test list

# View history
cm history

# Show documentation
cm docs
```

## Initialization for Agents

When initializing Checkmate as an agent:

```bash
# Always provide --url (non-interactive)
cm init --url http://localhost:8080

# Use --stealth for personal testing
cm init --url http://localhost:8080 --stealth
```

**Never** run `cm init` without `--url` in non-interactive mode.

## Creating Test Specs

### Basic Template

```yaml
name: "API Tests"
description: "Tests for [endpoint]"

requests:
  basic_request:
    body:
      field: "value"

tests:
  test_name:
    description: "What this tests"
    endpoint: /api/endpoint
    method: POST
    requests: [basic_request]
    expect_status: 200
    assertions:
      - query: "$[field]"
        expect: "expected"
      - query: "$[user][email]"        # nested access
        expect_type: string
      - query: "$[data][items].length()"
        expect_gte: 1
```

### Naming Conventions

- **Spec files**: `lowercase_with_underscores.yaml`
- **Test names**: `descriptive_test_name`
- **Request names**: `purpose_or_data_description`

### Common Patterns

#### CRUD Operations

```yaml
tests:
  create_resource:
    method: POST
    expect_status: 201

  read_resource:
    method: GET
    expect_status: 200

  update_resource:
    method: PUT
    expect_status: 200

  delete_resource:
    method: DELETE
    expect_status: 204
```

#### Error Cases

```yaml
tests:
  invalid_input:
    expect_status: 422
    assertions:
      - query: "$[error]?"
        expect: true

  unauthorized:
    expect_status: 401

  not_found:
    expect_status: 404
```

## Running Tests

### Output Format

Tests output JSON for easy parsing:

```json
{
  "suite": "User Tests",
  "summary": {
    "total": 5,
    "passed": 4,
    "failed": 1
  },
  "tests": [...]
}
```

### Interpreting Results

| Exit Code | Meaning |
|-----------|---------|
| 0 | All tests passed |
| 1 | Some tests failed |

### Verbose Mode

Use `-v` for detailed output:

```bash
cm test run -v
```

Shows:
- Each test being run
- Each request executed
- Each assertion result

## Debugging Failures

### 1. Check Failure Details

```bash
cm test run users -v
```

Look for:
```
✗ $[status]
  Expected: "active"
  Actual: "pending"
```

### 2. Review History

```bash
cm history
cm show cm-run-xyz
```

### 3. Verify Configuration

```bash
cm config show
cm config sources
```

### 4. Test Query Manually

```bash
cm clove eval '$[status]' '{"status": "pending"}'
```

## AGENTS.md Setup

Add to project's `AGENTS.md`:

```markdown
## API Testing

This project uses Checkmate for API testing.

### Quick Commands
- `cm test run` - Run all tests
- `cm test run <name>` - Run specific spec
- `cm history` - View test history
- `cm docs` - Full documentation

### Test Specs
Test specifications are in `.checkmate/tests/`.

### Creating Tests
See `cm doc examples` for test spec format.
```

## Best Practices

### 1. Check Existing Tests First

```bash
cm test list
```

Before creating new tests, understand what exists.

### 2. Run Tests After Changes

```bash
# After modifying API code
cm test run

# After modifying test specs
cm test validate && cm test run
```

### 3. Use Descriptive Messages

```yaml
assertions:
  - query: "$[status]"
    expect: "active"
    message: "User should be active after creation"
```

### 4. Test Error Cases

Don't just test happy paths:

```yaml
tests:
  success_case:
    expect_status: 200

  validation_error:
    expect_status: 422

  auth_required:
    expect_status: 401
```

### 5. Commit Test Specs

```bash
git add .checkmate/tests/
git commit -m "add(specs) user API tests"
```

## Discovery Workflow

When exploring a new API:

1. **Check for existing tests**
   ```bash
   cm test list
   ```

2. **Read documentation**
   ```bash
   cm docs
   cm doc examples
   ```

3. **Check configuration**
   ```bash
   cm config show
   ```

4. **Run existing tests**
   ```bash
   cm test run -v
   ```

5. **Review history**
   ```bash
   cm history
   ```

## Environment Handling

### Local Development

```bash
cm init --url http://localhost:8080
```

### CI/CD

```bash
CM_ENV__BASE_URL="$API_URL" cm test run
```

### Multiple Environments

```bash
# Development
cm test run

# Staging
CM_ENV__BASE_URL="https://staging.api.example.com" cm test run
```

## Common Issues

### "No .checkmate/ found"

Initialize first:
```bash
cm init --url http://your-api-url
```

### Tests Not Found

Check location:
```bash
ls .checkmate/tests/
```

### Connection Errors

Verify API is running and URL is correct:
```bash
cm config show  # Check base_url
curl http://localhost:8080/health  # Test connectivity
```

---

## See Also

- [Quick Start](QUICKSTART.md) - Getting started
- [Test Specs](TEST_SPECS.md) - Spec format
- [CLI Reference](CLI_REFERENCE.md) - All commands