checkmate-cli 0.4.1

Checkmate - API Testing Framework CLI
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85

---
description: Create a new test specification
---

# Create Test Specification

Guide for creating Checkmate test specifications.

## Test Spec Structure

```yaml
name: "Suite Name"
description: "What this test suite covers"

env:
  base_url: "http://localhost:8080"  # Optional, uses config if not set
  timeout_ms: 5000

requests:
  request_name:
    body:
      field: "value"
    headers:
      Authorization: "Bearer token"

tests:
  test_name:
    description: "What this test verifies"
    endpoint: /api/endpoint
    method: POST  # GET, POST, PUT, DELETE, PATCH
    requests: [request_name]
    expect_status: 200
    assertions:
      - query: "$[doc][field]"
        expect: "expected_value"
```

## Assertion Types

```yaml
assertions:
  # Exact value match
  - query: "$[doc][status]"
    expect: "active"

  # Type checking
  - query: "$[doc][count]"
    expect_type: number

  # Existence check
  - query: "$[doc][optional_field]?"
    expect: true

  # Comparisons
  - query: "$[doc][count]"
    expect_gte: 1
    expect_lte: 100

  # Array length
  - query: "$[doc][items].length()"
    expect_gte: 0
```

## Multi-Request Tests

For testing stateful behavior (counters, rate limits):

```yaml
tests:
  counter_increments:
    requests: [same_request, same_request, same_request]
    skip_first: true  # Don't assert on first request
    assertions:
      - query: "$[doc][counter]"
        expect_gte: 1
      - query: "$[@prev][counter]"  # Compare to previous
        expect_lt: "$[doc][counter]"
```

## Best Practices

1. One concern per spec file
2. Use descriptive test names
3. Include failure messages
4. Group related requests together