agent-runbooks 0.1.0

Convention and validation for AI agent runbooks
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95

# agent-runbooks Spec

## Commands

### audit
- Input: skill directory path
- Behavior: parse SKILL.md `## Runbooks` section, cross-reference against files in `runbooks/`
- Output: table of status (ok/missing/orphan), path, trigger phrase
- Exit code: 0 if all ok, 1 if any missing or orphan

### list
- Input: skill directory path
- Behavior: enumerate all `.md` files in `runbooks/`, extract title and line count
- Output: table or JSON (--json flag)
- Exit code: always 0

### validate
- Input: single runbook file path
- Behavior: check for title heading, steps section, numbered steps, broken links
- Output: errors and warnings
- Exit code: 0 if no errors, 1 if errors (warnings don't affect exit code)

### install
- Input: skill directory + template file path
- Behavior: copy template to `runbooks/`, optionally add reference to SKILL.md
- Exit code: 0 on success, 1 on failure

## Runbook Format

A valid runbook has:
- A title: `# <name>` (required)
- A steps section: `## Steps` (recommended)
- Numbered steps within the steps section (recommended)
- No broken relative links (required)

## SKILL.md Reference Format

Runbook references in `## Runbooks` section use:
```
- `trigger phrase` — [runbooks/filename.md](runbooks/filename.md)
```

The audit command parses lines containing `runbooks/` and ending in `.md` or `.md)`.

## Module Harness Integration

When a runbook contains a module harness header (Spec/Agentic Contracts/Evals), the validate command checks:
- Spec entries reference actual steps in the runbook
- Eval names follow snake_case convention

These are soft checks (warnings only, do not affect exit code).

## Agentic Contracts

### Following a runbook

When following a runbook, the agent promises to:
- Execute every step in order
- Not skip verification steps
- Report which steps were completed

### Validating runbooks

When validating runbooks, the agent promises to:
- Check for required H1 title
- Check for checklist section
- Check for imperative voice in steps

### Scaffolding runbooks

When scaffolding runbooks, the agent promises to:
- Never overwrite existing files
- Use bundled defaults only for missing files

## Evals

Planned evaluations for agentic contract compliance:

- `step_execution_order` — Does the agent follow steps in order?
- `verification_not_skipped` — Does the agent complete verification steps?
- `scaffold_no_overwrite` — Does `init_runbooks` preserve existing files?

See the [`evals/`](evals/) directory for eval definitions.

## Ontology Annotations

Runbooks can include ontology term annotations using inline markers:
```
[term:Domain] — references the Domain ontology term
```

The validate command accepts an optional `--ontology-dir <path>` flag. When provided:
- Scans the runbook for `[term:Name]` annotations
- Checks that each referenced term has a corresponding `.md` file in the ontology directory
- Warns on missing terms (does not affect exit code)