checkmate-cli 0.4.1

# Test Run History

Checkmate records every test run with git context for tracking and debugging.

## Overview

When you run tests, Checkmate:
1. Captures git state (commit, branch, dirty)
2. Records test results
3. Saves to `.checkmate/runs/runs.jsonl`

This enables:
- Tracking when tests started failing
- Correlating failures with code changes
- Reviewing test stability over time

## Viewing History

### Recent Runs

```bash
cm history
```

Output:
```
Test Run History (most recent first):

✓ cm-run-a3f 2024-01-15 5/5 passed @ abc1234
✓ cm-run-b7c 2024-01-14 5/5 passed @ def5678
✗ cm-run-c9d 2024-01-13 3/5 passed @ ghi9012*
```

Legend:
- `✓` = all tests passed
- `✗` = some tests failed
- `*` = dirty working directory (uncommitted changes)

### More Results

```bash
cm history -n 20
```

### Filter by Spec

```bash
cm history --spec users
```

### Filter by Commit

```bash
cm history --commit abc123
```

## Run Details

View full details of a specific run:

```bash
cm show cm-run-a3f
```

Output:
```
Run: cm-run-a3f
Time: 2024-01-15T10:30:00Z
Spec: users.yaml
Results: 5/5 passed, 0 failed, 0 errors (1234ms)

Git Context:
  Commit: abc1234
  Branch: feature/user-api
  Dirty: no
  Message: Add user validation
```

## Storage Format

History is stored in `.checkmate/runs/runs.jsonl` (JSON Lines format):

```json
{"id":"cm-run-a3f","timestamp":"2024-01-15T10:30:00Z","git":{"commit":"abc1234","branch":"main","dirty":false,"message":"Add user validation"},"summary":{"total":5,"passed":5,"failed":0,"errors":0,"duration_ms":1234},"spec_file":"users.yaml"}
```

### Record Structure

| Field | Description |
|-------|-------------|
| `id` | Unique run identifier |
| `timestamp` | ISO 8601 timestamp |
| `git.commit` | Short commit hash |
| `git.branch` | Current branch |
| `git.dirty` | Uncommitted changes? |
| `git.message` | Commit message (first line) |
| `summary.total` | Total test count |
| `summary.passed` | Passed count |
| `summary.failed` | Failed count |
| `summary.errors` | Error count |
| `summary.duration_ms` | Duration in milliseconds |
| `spec_file` | Test spec filename |

## Git Integration

### Tracking Test Stability

When tests fail, check history to find when they started failing:

```bash
# See recent runs
cm history -n 20

# Find the commit that broke tests
cm show cm-run-xyz
# Shows: Commit: abc1234, Message: "Refactor user service"
```

### Correlating with Code Changes

```bash
# Tests failed at commit abc1234
git show abc1234

# Or view diff
git diff abc1234^..abc1234
```

### Dirty Working Directory

The `*` marker indicates tests ran with uncommitted changes:

```
✗ cm-run-a3f 2024-01-15 3/5 passed @ abc1234*
```

This helps identify if failures are due to local changes.

## File Management

### Location

```
.checkmate/
└── runs/
    ├── runs.jsonl    # Run history
    └── .gitignore    # Ignores JSONL by default
```

### Default Gitignore

By default, run history is local (not committed):

```gitignore
# .checkmate/runs/.gitignore
# Run history is local by default
*.jsonl
```

### Tracking History in Git

To share history with team, remove the gitignore:

```bash
rm .checkmate/runs/.gitignore
git add .checkmate/runs/runs.jsonl
```

### Clearing History

```bash
rm .checkmate/runs/runs.jsonl
```

## Querying History

### With jq

```bash
# All failures
cat .checkmate/runs/runs.jsonl | jq 'select(.summary.failed > 0)'

# By date
cat .checkmate/runs/runs.jsonl | jq 'select(.timestamp | startswith("2024-01-15"))'

# By commit
cat .checkmate/runs/runs.jsonl | jq 'select(.git.commit == "abc1234")'
```

### With grep

```bash
# Find runs for specific spec
grep "users.yaml" .checkmate/runs/runs.jsonl

# Find failures
grep '"failed":' .checkmate/runs/runs.jsonl | grep -v '"failed":0'
```

## Use Cases

### Debugging Regression

1. Tests started failing
2. Check history: `cm history -n 20`
3. Find last passing run
4. Compare commits: `git diff <passing>..<failing>`

### CI/CD Integration

```yaml
steps:
  - run: cm test run
  - run: |
      # Check if tests are flaky
      cm history -n 10 | grep "✗" && echo "Recent failures detected"
```

### Performance Tracking

```bash
# See duration trends
cat .checkmate/runs/runs.jsonl | jq '{id, duration_ms: .summary.duration_ms}'
```

## Best Practices

### 1. Commit Before Testing

Run tests with clean working directory for accurate git context:

```bash
git add -A && git commit -m "WIP"
cm test run
```

### 2. Use Meaningful Commit Messages

Good commit messages help when reviewing history:

```
✗ cm-run-a3f 2024-01-15 3/5 passed @ abc1234
Message: "Add user validation"  # Helpful!
```

### 3. Periodic History Review

Check for flaky tests:

```bash
cm history -n 50 | grep "✗"
```

### 4. Track in CI

Export history as artifact for debugging:

```yaml
- uses: actions/upload-artifact@v3
  with:
    name: test-history
    path: .checkmate/runs/runs.jsonl
```

---

## See Also

- [CLI Reference](CLI_REFERENCE.md) - History commands
- [Hooks](HOOKS.md) - Trigger actions on test results