checkmate-cli 0.4.1

Checkmate - API Testing Framework CLI
# Assertions Reference

Complete guide to Checkmate assertions.

## Assertion Structure

```yaml
assertions:
  - query: "$[field]"            # Clove query expression
    expect: "value"              # Expected value (various types)
    message: "Failure message"   # Optional description
```

## Assertion Types

### Exact Match (`expect`)

Match exact values:

```yaml
# String
- query: "$[status]"
  expect: "active"

# Number
- query: "$[count]"
  expect: 42

# Boolean
- query: "$[enabled]"
  expect: true

# Null
- query: "$[deleted_at]"
  expect: null

# Array (exact match)
- query: "$[tags]"
  expect: ["a", "b", "c"]

# Object (exact match)
- query: "$[config]"
  expect:
    key: "value"
```

### Type Check (`expect_type`)

Verify value type without checking exact value:

```yaml
- query: "$[id]"
  expect_type: string

- query: "$[count]"
  expect_type: number

- query: "$[active]"
  expect_type: boolean

- query: "$[items]"
  expect_type: array

- query: "$[metadata]"
  expect_type: object

- query: "$[deleted]"
  expect_type: null
```

### Existence Check

Check if a field exists using `?` suffix:

```yaml
# Field should exist
- query: "$[required_field]?"
  expect: true

# Field should not exist
- query: "$[forbidden_field]?"
  expect: false
```

### Numeric Comparisons

Compare numeric values:

```yaml
# Greater than
- query: "$[count]"
  expect_gt: 0

# Greater than or equal
- query: "$[count]"
  expect_gte: 1

# Less than
- query: "$[errors]"
  expect_lt: 10

# Less than or equal
- query: "$[errors]"
  expect_lte: 5

# Range (combine multiple)
- query: "$[percentage]"
  expect_gte: 0
  expect_lte: 100
```

### Compare to Previous Response

In multi-request tests, compare to previous response:

```yaml
# Current greater than previous
- query: "$[counter]"
  expect_gt: "$[@prev][counter]"

# Current equals previous plus one
- query: "$[version]"
  expect: "$[@prev][version]"
```

---

## Query Expressions

### Basic Navigation

```yaml
# Field at root
- query: "$[status]"

# Nested field
- query: "$[user][profile][name]"

# Array index
- query: "$[items][0]"

# Array index then field
- query: "$[users][0][name]"
```

### Array at Root

When the response is an array directly:

```yaml
# Response: [{"id": "a"}, {"id": "b"}]
- query: "$.length()"
  expect: 2

- query: "$[0][id]"
  expect: "a"
```

### Array Methods

```yaml
# Array length
- query: "$[items].length()"
  expect_gte: 1

# First element
- query: "$[items].first()"
  expect_type: object

# Last element
- query: "$[items].last()[id]"
  expect_type: string
```

### String Methods

```yaml
# Uppercase
- query: "$[code].upper()"
  expect: "ABC"

# Lowercase
- query: "$[name].lower()"
  expect: "john"

# Length
- query: "$[name].length()"
  expect_gte: 1
```

### Scopes

```yaml
# Current response
- query: "$[field]"

# Previous response (multi-request tests)
- query: "$[@prev][field]"
```

---

## Failure Messages

Always include messages for clarity:

```yaml
assertions:
  - query: "$[status]"
    expect: "active"
    message: "User status should be active after creation"

  - query: "$[balance]"
    expect_gte: 0
    message: "Balance cannot be negative"

  - query: "$[items].length()"
    expect_gte: 1
    message: "Response should contain at least one item"
```

Messages appear in test output on failure.

---

## Common Patterns

### API Response Structure

```yaml
assertions:
  # Success response
  - query: "$[success]"
    expect: true
  - query: "$[data]?"
    expect: true
  - query: "$[error]?"
    expect: false
```

### Pagination

```yaml
assertions:
  - query: "$[page]"
    expect: 1
  - query: "$[per_page]"
    expect: 20
  - query: "$[total]"
    expect_gte: 0
  - query: "$[items]"
    expect_type: array
```

### Created Resource

```yaml
assertions:
  - query: "$[id]"
    expect_type: string
    message: "Should return resource ID"
  - query: "$[created_at]?"
    expect: true
    message: "Should have creation timestamp"
  - query: "$[updated_at]?"
    expect: true
```

### Error Response

```yaml
expect_status: 422
assertions:
  - query: "$[error]?"
    expect: true
  - query: "$[error][code]"
    expect: "VALIDATION_ERROR"
  - query: "$[error][message]"
    expect_type: string
  - query: "$[error][details]"
    expect_type: array
```

### Counter Increment

```yaml
requests: [request, request, request]
skip_first: true
assertions:
  - query: "$[count]"
    expect_gt: "$[@prev][count]"
    message: "Counter should increment"
```

### Rate Limiting

```yaml
requests: [request, request, request, request, request]
assertions:
  - query: "$[remaining]"
    expect_gte: 0
  - query: "$[remaining]"
    expect_lt: "$[@prev][remaining]"
    message: "Remaining should decrease"
```

### Authentication Token

```yaml
assertions:
  - query: "$[token]"
    expect_type: string
  - query: "$[token].length()"
    expect_gte: 20
    message: "Token should be at least 20 characters"
  - query: "$[expires_in]"
    expect_type: number
  - query: "$[token_type]"
    expect: "Bearer"
```

### Nested Object Validation

```yaml
assertions:
  - query: "$[user][id]"
    expect_type: string
  - query: "$[user][email]"
    expect_type: string
  - query: "$[user][profile][avatar]?"
    expect: true
  - query: "$[user][settings][notifications]"
    expect_type: boolean
```

### Array at Root (List Endpoints)

```yaml
# GET /api/items returns: [{"id": 1}, {"id": 2}]
assertions:
  - query: "$.length()"
    expect_gte: 0
    message: "Should return array"
  - query: "$[0][id]?"
    expect: true
    message: "First item should have ID"
```

---

## Assertion Failures

When an assertion fails, output includes:

```json
{
  "failures": [
    {
      "request_index": 0,
      "assertion": "$[status]",
      "expected": "active",
      "actual": "pending",
      "message": "User status should be active"
    }
  ]
}
```

### Debugging Tips

1. **Run with verbose**: `cm test run -v`
2. **Check actual value**: Look at `actual` in failure output
3. **Verify query path**: Use `cm clove check` to test queries
4. **Check response**: The full response is in the JSON output

### Common Errors

**"Cannot use string key on array"** - The response is an array at root level. Use `$[0]` or `$.length()` instead of `$[field]`.

**"Query returned null"** - The field doesn't exist. Check the path or use `$[field]?` for existence check.

---

## See Also

- [Clove Queries]CLOVE.md - Query language reference
- [Test Specs]TEST_SPECS.md - Full spec format
- [Examples]../tests/ - Real test examples